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Introductory Note 



This was the fourth conference to be convened by CIET on behalf of 
the Committee on Research and Development in Modern Languages. Like 
others, it brought together a number of cxj>ei ts to discuss a subject of immediate 
interest to language teachers. It aimed to survey and clarity opinion about exist* 
ing language examinations and possibly to indicate topics on which research 
might be useful. 

In some form, examinations affect almost all language teaching and most 
teachers have strong views about their influence or efficiency. Only limited 
aspects of the held could be discussed in the time available: these asj>cets 
included GCE and CSE examination*, ccitain non-school examinations, 
examinations in English as a foreign language and the use of objective techniques 
and continuous assessment. University level examinations were not sfjecificaUy 
considered. 

ILe following chapter* include revised and edited versions of papers 
presented, a postscript arising from one session (Chapter ft), and a summary of 
the main opinions in the discussions (Chapter 9). The opinions expressed in the 
papers arc of course those of their individual authors. Apixmdiccs provide a note 
of relevant research known to be current in Britain, a select bibliography and a 
list of the participants. 

C. E. Perron 
Director , 

Centre for Information 
on language Teaching 
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Boards examining for the General Certificate of Education { GCl C) 

Associated Examining Board AEB 

University of Cambridge lx>cal Examinations 

Syndicate Cambridge 

Joint Matriculation Board (Tbc Universities of 
Birmingham, Leeds, Liverpool, Manchester and 
Sheffield) JMB 

University Entrance and Schools Examinations 

Council, University of london London 

Oxford and Cambridge Schools Examinations 
Board O & C 

Oxford Delegacy of lxx:al Examinations Oxford 

Southern Universities’ Joint Beard for School 
Examinations (Universities of Bath, Bristol, 

Exeter, Reading, Southampton and Surrey) SU |B 

Welsh Joint Education Committee WJEC 
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Education Committee Nor them Ireland 
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Ixrndon Chamber of Commerce LCC 

Royal Soc u ty of Arts RSA 
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CHAPTER i 



The influence of GCE examinations 
on teaching modern languages 



(J) ‘ O' level 

B. \V. PACE 

Wc all know, of course, that examinations should not influence teaching 
methods at all. Wc, the teachers, should aim at teaching our subject to the 
highest level of coin|>etcmc our individual pupils can attain and external public 
examinations should arrive as an incidental, objective assessment of that attain- 
ment. Wc all know, equally well, that this is not true. Probably the most 
undesirable result of the wholly justified dcmocratisalion of education over the 
past twenty-five ycats has been the enormous itnj>oitancc assumed by examina- 
tion results. In a society increasingly oj>cn to all the talents, where people arc to 
be selected for higher education, the civil scivicc, banking, insurance 01 what you 
will, readily available criteria will lx? used and public examination results will 
provide them. Whatever our aims in teaching a 'foreign' language -some com- 
|>clcncc in understanding, shaking and reading, some awareness of French 
culture, a feeling of forcignncss, of the validity of a different view of life- one 
of our aims is undoubtedly io get our pupils through 'O' level. To that end we 
study the examination, we try to sec precisely what the examiners arc testing, we 
even draw up syllabuses based on past examination paj>crs (‘no need to teach 
the subjunctive- it never comes up*) and wc teach accordingly. Some embrace 
this situation with ciithmiaun, others less wholeheartedly, but wc all feel that 
whatever our private opinion of die examii .non, ovir duty towards our pupils 
is to prepare them for it to the best of our ability. 

Ihc traditional 'O* level examination 1 of comf>osit 3 on, translation from 
and into the language, and [K*sibly some sort of reading or aura! comprehension 

1 I mention French only in this r*per. There i» very little difference in the treatment of 
other modern language* by the examination boards 
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test or reproduction, tries to give something to both the reformers "“ho insist 
that in order to learn French you must use French, and the traditionalists who 
insist that, for French to have any educational justification, pupils must show 
that they know it by being able to translate (particularly into French) and show 
their intellectual paces thereby. 

The oral part of this examination generally consists of a reading test and 
a few minutes' conversation prompted by some elementary questions on age, 
weather and the reading passage. Over the past six years, various boards have 
produced alternative examinations which do away with translation into French 
or make it an optional alternative to a reproduction or comprehension test 1 . 

The traditional examination, however, which is still taken by far the 
largest nutnlx'r of candidates', tests translation above all else. The normal Joint 
Matriculation Board French ‘O* level 1 , taken by over 30,000 candidates, may 
serve as a typical example. In it, 45 per cent of the marks arc given for writing 
Fmglhh and another 20 per cent for translating from English. Twelve per cent arc 
given for reading or writing given French (dictation and reading passage). Only 
23 ]>er cent arc given for individual written and sj>okcn expression in the 
language. 

Translation, then, and particularly translation into French because it is 
the most difficult, is dominant. In vain e\)>cits tell us that an oral approach, 
even in these conditions, is nevertheless the most (fleet ivc. We do not dare to 
believe this because in the experience of most of us it is not true. We teach our 
pupils to rcs])Ond in structured situations like this : 

CVst toi qui as donn<5 le ballon \ Jean- Paul ? 

Ouh je le lui ai donn£ 

or : Non, eVst Michel qui le lui a donne etc. 

We can sometimes get this sort of phrase used spontaneously in an vmstmetured 
situation- certainly we hope to. But this is no guarantee at all that the same 
pupil faced with the English : 

I gave it to him 
or even worse : 

I gave him it 

"ill get the answer right. The situation is quite different; the pronoun it is no 
longer referring back to a previously mentioned It ballon nor lui to d Jean-Paut. 



'St< Not e f, p. 14 
* Sti Note II, p. II 
'Sr* Note III, p. 15 
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The pupil now has to make a quite different set of connections between 1 / and le 
and him and lui and all this has to be taught. Hence the fiendishly ingenious 
sentences we concoct : 

Why did he give some of it to them thci e ? 
which bear no relation to real live communication any more than do many 
language laboratory drills. 

Most boards go to considerable trouble to involve teachers in the formu- 
lation of their examinations so that it may be claimed that tests reflect the 
method.-, of teaching rather than distort them. Whether they reflect or distort is 
a chicken and egg problem and depends on the individual teacher. If he is a 
traditionalist, the normal ‘O’ level reflects his teaching; if he is not, it distorts 
it. However, public examinations have tended to reflect the more conservative 
methods and the new alternative ‘O’ levels result from discontent over (he years 
of a growing minority who want to use different methods, Nevertheless, the 
traditional examination has not had by any means a totally stifling effect. All 
the audio-visual, audio-lingual, language lalioratoiy courses and methods of the 
past ten years have been initiated under its domination and most have had to 
justify themselves according to its criteria 1 . But these cx|>ci intents have been 
carried out by enthusiastic teachers who were willing to take a risk with their 
pupils; many who arc sympathetic to new methods have not embarked uj>on 
them, and uj>on the often huge capital investment involved, purely because there 
was, until recently, no suitable examination at the end of the course. 

'I he alternative pa|>crs now set b) several examination Ixtard* vary but 
all have the same characteristic of abolishing translation from English and of 
placing more ctnphads on shaking or reading. I be JMB ‘O' level French 
(Alternative Syllabus) 1 , which is very similar to the Catuhiidgc Alternative B, 
goes furthest in this direction. Ill it only 10 per cent of marks are given for work 
in English, 30 j>er cent for passive com] ire be ns ion, 25 ]*r cent for srmi-diicctcd 
use of French (answers in Ficnch to given questions) and 30 per cent to indi- 
vidual expression in the language. I his new examination presents m with a 
new set of eriteiia to satisfy. Ilic arguments of those ‘who me prepared to make 
some concession to what they feel vaguely to lie a modem view and who dee lire 
that they use the “direct method** in the fust )car but arc then compil'd by 
the require merits of the public examination* to “get down to work**, the iinplica- 

1 dVe S, R. Ingram and ). Mace, 'Audio -visual Fi<n(h — fuithcr f*poicncc\ 
btfUjCWjtM, sol. XI, IV, 196b for how the first elates ‘Tatted r*n an audiovisual te>ur>e 
vt re rxpeeled lo perform at ‘O' level 
Note IV, p lb 
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lion being that what was being done in the first year was not really work and 
that grammar and translation alone are work’’ ean no longer stand, French 
sj>okcn and written without recourse to English is now required, *lhc aim of 
tliis syllabus is to evaluate attainment in speaking and writing French and in 
comprehending its sj>okcii and written form, reducing to a minimum the role 
of English* 2 . What used to be considered ‘frills* arc now ‘real work*, i.c. what the 
examiners want. Candidates must read a large amount of varied material, 
must hear a large number of different native voices and mint have an immense 
amount of individual practice in speaking — 65 per cent of lire marks arc given 
for the use of these skills. Moreover, 25 per cent go to individual sjiokcn rcsj>onsc 
--different for each of the thirty or so members of a class. Time must also be 
found for noimal practice in writing. How can such skills lx' taught, tested, 
guaranteed accurate and yet retain some individual sfjonlamity ? We have all 
experimented with beginners in F.encli. We all know bow delightfully primary 
school children can go through their dialogues, how even up to their second 
and third year in secondary school, children can retain some skill in conversing 
in French even when they have given up conversing in English, hut most of us 
have not until now tried to make evety single member of a fifth foim able to 
converse in French for five full minutes, knowing that Ins |xr foimmcc will 
count for a quarter of bis total marks. 

All this needs a fundamentally different approach. A method in which 
a traditional ccxirsc lxx»k is followed year by sear with little or no me of a 
tape recorder, and only the occasional additional reader, will not prepare can- 
didates for this examination. We are no longer in the jxisilion of having to 
make the difficult transition to an examination-based liook-coufsc after two or 
three )cars of oral method. We can go on using the techniques already often 
used in the first two >ears without the nagging feeling that the skills we are 
teaching arc going to l>c largely irrelevant to the examination. Group work 
which progress's through the years from conversation drills on s|>ccific joints 
or directed dialogues to a more genuine free exchange of ideas and vic\>s on 
topics and book* Ixing studied, is now directly relevant to a major part of the 
examination. Almost an) thing can feed this group in its later stages : the pupils' 
own everyday txfxricncfs, last Saturday's match, the l»ook$ they arc reading 
in French or English, a topie or project the) are engaged uj»on. news of their 
Flench correspondent* and the making of taj>cs to send to their 1 rcnch con- 
tacts. All this ran In' the subject of question and answer and dimension in 
small grou;* of four or five. Class libraries of readers covering as l uge an area 

1 T. M. Hodgson, l.fcming rWoa Unevoett. Rouilcdge, 1955, p.7 
1 Preamble to the regulation* for the Alternative Syllabus in 'O* lewl french. JMB 
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of interest as possible should provide individual reading and the basis for much 
written work, if work sheets consisting of questionnaires, some contextualised 
drills and directions for summaries, reproductions and guided coinj>o$itioii$ 
accompany them. 



Conversation ( 0 ) 

The ‘hundred questions* technique ha<bcen subjected to much criticism. 
It is a laudable attempt to chart an area of the examination in advance. A hun- 
dred questions on all sorts of subjects arc set and are prepared for during the 
course of the year. In the examination ten arc chosen at random for each can- 
didate, who is expected to answer without hesitation or need for much reflec- 
tion. This has been criticised as too mechanical a test involving too much 
rote-learning, but it can be as mechanical or not as the teacher or candidate 
wishes. The questions (some of which could be improved) cover a great deal of 
gro* .<d and can be the basis of much conversation practice in groups or with 
the whole class. Since they ingeniously involve a good variety of tenses ilicy 
arc invaluable for provoking meaningful pattern practice in verb forms and 
tense usage. 



Conversation (6) 

This has to be a real conversation and not a memorised lecture. Five 
minutes seems a long lime to the prosj>cclivc candidate but, in practice, he 
usually has little difficult)- in Tilling it. ’I he transition in group or class work 
from surcotvjied drills, dialogues and role-playing to freer exchange is very 
difficult to make but has to be made if the language i< to become a genuine 
means of communication. Ibis test obliges teacher and pupils to make the 
effort . It has been argued that too much reliance on predetermined conversa- 
tion material can in fact hamper the subsequent leap to indcjKndent com- 
munication. If this is true, the test will encourage us t > wean our pupils from 
drills. 

1 he use of an aud' 0 -visual course and graded taf>cd inatciial improves 
listening comprehension quite dramatically. It is entirely appropriate that the 
examination should encourage this by giving it a substantial number of marks, 
and it is not surprising that the examiners have found this test to be well done. 

The abolition of translation has produced a distinct improvement in the 
authenticity of written French, Anglicisms still occur but the habit of thinking 
i* all out in English first is dying. 



Fluency in all the skills of hearing, speaking, reading and writing is 
much improved by the training imposed by this examination. But mistakes still 
abound and the old conflict between fluency and accuracy has been intensified 
rather than resolved. Discovery methods, ‘working jt out for oneself, arc not 
generally relevant to modern language teaching. We cannot alford to let our 
pupils acquire approximately correct linguistic habits even if they can com- 
municate adequately with them, Wc are always having to make the difficult 
judgement of when to interrupt the flow in order to correct the grammar, 

'I he examination is by no means perfect yet and there arc several ways 
in which it could be rapidly and easily improved. Above all, the board should 
accept the responsibility it undoubtedly lias in influencing what is done in the 
classroom. Ihe new JM1I syllabus docs not merely evaluate attainment; it 
encourages certain skills in a quite specific way and it should not avoid tl 
ensuing responsibility. This lies in two main areas : 

(1) Accuracy: 'ITc word occurs again and again in the regulations and 
it is assessed and marked in all the forms of written and qxakcn 
expression but nowhere is it defined. When the members of a com- 
mittee of the 1 ncorjxuatcd Association of Assistant Masters were 
revising The Teaching of Modem Languages in 1007, they wrote 
to all the examination lxwrds to ask their views on the list of gram- 
matical toUrances published by the Ministerc dc l‘l instruction Fub- 
liquc in KOI and gave a list of those alternatives they considered 
valid today. Only three of the lx>ards rcp’ied and they presented 
such divergent views that nothing could be decided. If the limits of 
tolerance on joints of grammar arc so fluid, how much more so arc 
(hey in pronunciation, accent, intonation, fluency and other intcr- 
dcfxndcnt qualities, all of which arc judged and marked in this 
examination. Such published scales for assessing taj>cd examinations 
as exist 1 ate scarcely helpful. Each quality described as markable— 
angl : ciscd, laboured t inconsistent p uncertain, fairly accurate — is 
purely subjective. How arc wc to know that our views arc the same 
as the examiners*? Ihc answer seems simple enough. After a given 
examination a sample half do/cn written compositions should he 
photo-copied with a commentary on how the marks were awarded. 
For conversation and reading lest a similar sample of tapes should 
be made. All these would be available to any teacher who wanted 
to find out to just what standards he was supposed (o lx* working. 

'Otter, II. S. ? 'An oral proficiency asxMmcnt chart*. Medan lunevaeff, \xA. XLVI 
no, I, December 1965 
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(2) Linguistic content: 'Hus [>oint has been made many times before, 
but if there is a difference between 'O’ level Trench and ‘A 1 level 
Trench, it should be more closely defined. The new ‘Nuffield’ 
examinations, where 75 per cent of the linguistic material will have 
appeared in the course itself, go some way towards this. 'The boards 
setting new examinations must define the linguistic areas in which 
'vc arc sup]>oscd to be working. 

Some details of the new JMB examination could be improved. Reading 
a passage aloud seems a highly artificial activity. If reading aloud is necessary, 
I should like to sec a printed dialogue of which the examiner reads one part 
and the candidate die other. 'Jins would fit in much better with the give and 
take of modern methods. 

'1 he comjjosition could be modified. 'The pictures are sometimes tire* 
somcly facetious and ihc boards should perhaps suggest different styles in which 
the story could be retold : a folieemans report, a journalist's account for his 
paper, a television newsreel interview with one of the participants or the 
incident recounted as past of a play or novel. 

'llic examination should include some work done during the course of 
the year. What can be done on a particular day is by no means necessarily 
one’s best f>crformance and some pieces of class work done at different stages 
would help to shorten the odds against the nervous, lx>rdcrlinc candidate who 
is j>cculiarly vulnerable, 'lbc new alternative ‘A’ level offered by the JMB for 
1972 includes an element of this sort and it could profitably be included at 
"O’ level also. 

One other important skill has never been tested by examination hoards 
- fa<{ reading. Mr*t teaching of modem languages, even using new methods, 
is still so concerned with grammatical accuracy that, although much more 
reading is done than before, there is no pressing need for the pupils to progress 
beyond the stage of slow subvocalisation or even mental translation when read* 
ine. If a tc^t were included which involved the reading and romprehrnonn of 
passages of French too long to translate within the time available, the acquisb 
tion of a valuable skill world be encouraged. 

One cannot, however, expect a single examination to do everything. 
Ibe JMB Alternative 'O’ level in French and the Cambridge Alternative B, 
MKtre revolutionary in conception and execution than the minor alterations 
practised by some other Ixxards, already do much. ITierc is, however, rc*om for 
still more alternafiscs to provide relevant objective#. 
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NOTE 1 

Alternatives to translation into French offered in J970 



AEB: 


Written comprehension test with French ques- 
tions requiring answers in French; larger con- 
versation. 


c Cambridge : 


Alternative B: A second free composition; a much 
enlarged oral/aural test (as for JMB Alternative). 
Alternative C: Written comprehension test with 
French questions requiring answers in French; a 
more difficult dictation. 


JMB: 


Written comprehension test with French ques- 
tions requiring answers in French; a much 
enlarged oral/aural test (see Note IV). 


London: 


Written comprehension test with French ques- 
tions requiring answers in French. 


Oxford: 


Aural comprehension test requiring written 
answers in French. 


O & C: 


Free comjMasition from pictures; larger conversa- 
tion. 


SVJB: 


Reproduction of a spoken .tory of 230 to 250 
words. 



* In 1971 Cambridge is combining its alternatives into one s>llabus so that 
various choices of test can be made. The translation into French will be an 
alternative to a second free conq>osition. 



NOTE II 

Numbers of candidates [where m ailable), Summer 1909 



Candidates offering translation into French : 




Cambridge ... 


1 3,027 


JMB 


30.400 


London ... 


28.783 


Oxford 


21,921 


o & c 


10.042 


U’JEC 

•Summer 1900. 1909 figures not yet available. 


0.575 



o 
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2. Candidates offering alternatives to translation into French : 



A Eli 8,886 

Cambridge ... ... ... ... ... ... 4,635 

JMB 5,687 

Oxford 2,876 

3. Percentages offering translation or alternatives in Ixxnds supplying 
both : 

Translation Alternative 

Cambridge ... ... ... ... . . 77 23 

JMB 86,5 13.5 

Oxford ... ... ... ... ... 88.4 11.6 

NOTE III 

JMB ( 0' level French Syllabus (traditional form) Allocation of Marks 

Paper l: 

1. Translation from English ... ... 20% 

2. Free comjosition — a story based on a given set 

of pictures or outline; or a letter ... ... 16.6% 

Dictation 8.3% 

Paper 2: 

1. Two passages for translation from French ... 30% 

2, Reading comprehension test : written English 

questions requiring ansvc's in English ... 8,6% 

Aural Comprehension test: Written English ques* 

tions rcqviiring answers in English ... ... 6.6% 

Oral Examination: 

1. Heading Test ... ... ... ... ... 3 3% 

2. Conversation ... ... ... 6.6% 



NOTE IV 

JMB 'O' level French (Alternative Syllabus) Allocation of Marks 

Paper /: 

1. Fur tuhij *»Mtnjn , will!) sjniil.il to the Uadi* 

tional form ... ... ... ... ... 15% 

2. Reading compTchenrion test: written French 

questions requiring answers in French ... 15% 

Paper 2: 

I. Listening comprehension test : a three-part 
likening test on lipe lasting thirty minutes 
with multiple choice answers. 
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Part i: the candidate hears a remark and lias 
to choo.e who the speaker is or where he is. 

Part ii: the candidate hears a scries of con- 
nected rcmaiks or questions on a theme and 
has to choose appropriate answers or comments. 

Part iii: two short incidents arc related and 
the candidate chooses appropriate statements 
of fact about them ... ... ... ... 15% 

2. Reading comprehension test : a two-part test 
with multiple choice answers. 

Part i : a series of single sentence statements 
a!>out a person in a given situation. The candi- 
date chooses what is likely to he said. 

Part ii: four prose passages on various subjects 
arc given; the candidate chooses statements of 
fact about them ... ... ... ... 15% 

3. Translation from French (the first of the two 

passages on the traditional paper) ... .. 10% 

Oral Examination: 

1. Reading Test 5% 

2. Commotion (a): the candidate prepa res before- 

hand the answers to a hundred given ques- 
tions on a variety of topics. Ten are chosen at 
random in the examination room for the can- 
didate to answer ... ... 10% 

Commotion (6); the candidate converses with 

the examiner on two books chosen at random 
from three prepared or on one topic chosen 
from four prepaid ... ... 15% 

(2) 'A' level 

\V. M. S1IORT1' 

Composition of sixth fount taking modern languages 

Today’s problem is not so much one of increased numbers but of the 
lack of homogeneity cf the sixth form. PupiU choose sixth form courses for the 
following reasons (not necessarily in this order of importance) : in the junior 
and middle school, sympathetic leaching and/or the individual jvrsonality cf 
a teacher have created an interest and liking for more advanced work; certain 



subjects offer special interest and satisfaction; [rest ‘O' and CSE level educa- 
tion with, perhaps, a further qualification, may lead to a rewarding career. 

Thus a foreign language sixth form today, particularly in some well- 
established urban comprehensive schools, could consist of the following clearly 
defined groups : 

(!) Pupils with good ‘O' level grades, probably going on to university 
or similar institutions, who can cope with and derive benefit from 
the existing *A* level examination. 

(2) Pupils with indiffcrent/[x>or ‘O’ level grades, but who still wish to 
continue their foreign language, using it possibly as a third subject 
supporting their other two ‘A 1 levels. Ibis group needs a revised 
syllabus and a new examination. The proposed minor examina- 
tion (Schools Council Working Paper 5) would have been very 
Suitable. 

It is interesting to note that, commenting on the proposals to 
establish GCE in 1951, the Cambridge and O & C boards regretted 
the pacing of the Higher School Certificate Subsidiary Level (as do 
many of the older teachers). The steady rise in |>c»pnlarity of the 
‘O’ (‘A 1 ) subjects winch these boards offer for examination at a level 
between ‘A’ and ‘O’ level indicates the need for this type of examina- 
tion. It is a pity that passes at (his level arc recognised as being only 
‘O’ level. 

(3) Tost— CSE pupils, interested in the life and customs of a foreign 
country but with no film skill in the written language and with 
varying degrees of competence in shaking it. Their command of 
the printed word will be passive rather than active. 

(4) Pupils beginning a foreign language and needing intensive prepara- 
tion. Kor these, ‘O’ level is hardly sophisticated enough and existing 
‘A’ level plainly will not do. 

Even if groups 1-3 arc arranged in sets according to ability (not always |<»s* 
sible because of subject grouping) they still face the same examination at the 
end of their course : an examination based almost exclusively on active use of 
the foreign language with literary studies added, and whose sco]>c and form 
have remained remarkably stable throughout a period of gTcat educational 
change and upheaval. 

Xotutc of 4 A' trvft 

Whatever one considers 'A 1 level to be — a leaving examination to test 
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work done in language and literature, or a device for pacing information on 
to universities about their |>ros|>ective entrants— this examination determines 
not only the nature of a sixth form course, but also the significance attached to 
different elements of the course. Good teaching techniques arc not necessarily 
synonymous with good examining techniques, but in practice the majority of 
teachers do teach to this examination, 'faking into account the difficulties faced 
by candidates applying for the Arts faculties of universities, the) cannot do 
otherwise. The |>crccntagcs conventionally awarded to the middle range, grades 
B to 1) in particular, arc so tightly grouped that borderlines become all imj>or* 
tant. B, C, C would probably secure for a candidate a place at the university 
of his choice, whilst B, C, I) may well not. Moreover, many t>oards have ‘hurdles’ 
in various sections of their examination : c.g. in French, Ixmdon, Northern 
Ireland, SUJB and Oxford im|)Ose hurdles on the language papers and to these 
Northern Ireland and SUJB add the oral test. ARB and O & C require a mini- 
mum |>ci formancc in the prose and essay. Whatever U adopted -i.c. the iinj>os* 
ing of hurdles or the allocation of marks to give a balancing or weighting clfccl 
— this will inevitably be reflected in the emphasis which a teacher gives to the 
different elements in his course. 

1'rnrulotion into the jorcign fonguagr 

In recent times, prose translation lias had to withstand fairly sustained 
attacks. Ifiesc can be summarised in the argument that it is of doubtful value 
as a means of learning a foreign language and preparing for it wastes much 
valuable time that could be better s|>cnt in building up experience. In spite of 
these criticisms, I hold the view that it is a valid test for the university aspirant 
to undergo, but witii the proviso that the passage can be safely rendered into 
the foreign language, avoids undue sophistication of idiom, literary or highly- 
coloured vocabulary and is within the pupil’s exj>eiicncc. 

The following examples of what is considered inappropriate are taken 
from recent JMB papers for translation into Spanish : 

(1) or a beggar, flopping at home aftrr house, who raised his voice 
At each ttja, the wrought iron gate that led into the fatio, and begged 
for alms with the phrase of inline moti.il u*acc.* 

(2) 'She had the black hair of the true G>p'> and an indolent terrible man- 
ner as though driving prisoners to their execution, and 1 shou’d have 
written 'criminals*, for the instincts of such a creature were surely 
upon the side of crime.’ 

(3) “Go on and read a book all full of love affairs with the beautiful shiny 
black princesses.’ 

(4) 'The warmth of the sun fell e>n us like a treasure, and the daslirht 
moved over die sea in gieat. sbw traMposiikn* of colour, dicing each 
night in purple dusks. The cliffs soaked up ihe sunsets like red sponges 
and the distant mountains shone blue at a gigantic saw.* 
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It is true that during the first year of the VV level course training pupils 
in prose translation is often a rather tedious affair. But after iwo terms at the 
outside, it need not take up more than one period per ten days, and then purely 
for the purpose of correcting. Any teacher who gives prose lessons regularly 
over two years is simply wasting his own and Ins pupils* time, 

'translation into English 

Too frequently this turns out to be a literary, stylistic exercise. Here, 
possibly more than anywhere else in the VV level examination, are the tradi- 
tional university approach and influence apparent. The candidate is faced with 
passages of admirably descriptive writing — 'the storm*, 'a fishing village’ , ‘a 
rural scene’, ‘a harbour scene and village* — arc representative titles of some of 
the passages set in Spanish by the JMB over the last decade. (How refreshingly 
different was the passage of 1 9G8 about Berlin, written in a good, journalistic 
style!) I much prefer my pupils to have to deal with a piece of French of this 
sort : 



'Quarante pour cent dc* ^tudiant* frmciis font un travail r^mun^r6 chaque 
innee, Celt rfirnportc quoi, Tout CM bon. L'osrnticl c*l que lapporie 
do sou*, {vidttnmcnt. Pat example, l iroii heure* du matin on enfile une 
blouse i ia gate dc Vaugirard ct on prend k pleini brat cinquanlc poult* 
plunufci cl video qui viennert d’arriver dc* quatie eoim dc la Bretagne. 
El on lei charge *ur un camion, he camion plcin, on rnonte dim la cabine 
ct on donarre. CVil beau, Pari*, dam Ic petit rnaiin. Anivc *u* Hallo, on 
d^charge 1c camion. Puii on rcpari pour fa gare dc Vaugirard cl ga rceom- 
mencc. II ne faut pa* flctfiir: on ic ferait rtrnplaccr par lei ‘vraii’ porlcuri 
On fait ca pour dix franc* la nuil. Parfoii on a le droit k un ca**e -croCtc 
«n lupplemcnt: Camembert, pain, un ^uart dc sin. Aprf* $», tn icnlrant, 
il faut plongsr le ne/ dans hi bouquim ct hi coun. Bicn *ur, il y a do 
boogne# mom* tudes; jcnir dc t^moin i un procc'i; fairc fc quatrirnie dan* 
un lournoi de bridge: prornener dc* cnranli faprf *-rniai. Mai* cV*t 
Etscepiionncl,* {AEB f Soi tmbtt 1967) 

l or them, at their level, it i* much more authentic than this : 

‘Ricn n'ui plus d^Iicicux que cc* premiere* joumlcs d'autnmnc ou Pair 
agiti de pui**ants remous scmblc une mcr invisible donl le* vague* *r bri*cnl 
dar* Ic* arbre*, tandi* que Ic lolcil, dominant cette fuirur ct ce tumuhe, 
accorde k la moindre ficur 1‘ombrr quVIle fera loumcr i Kn pied ju«qu*au 
loir. 1 if MB. 1967) 



Essay 

Here we arc faced with ambiguity, imprecision of purpose or aim. On 
the one hand, it is clear from many of the topics set that the essay is viewed in 
its tradional light — a piece of airy, cntcitaining nonsense, a didactic com- 
panion, or a discursive argument with the assumption that the writer has at 
his dhpxwal a wealth of vocabulary and a command of stylistic idiom. Vet, on 
the other hand. GGE l>oards f though not alb, allocate substantially more marks 
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to form than to content, thus indicating what they hold to be more important, 
i e. correct use of the foreign language. 

Now the average sixth form pupil finds his command of the foreign lan- 
guage haidly adequate to do himself justice in this test. The distance separating 
his achievement in his own language from what he can realise in the foreign 
language is very great indeed— far greater than it was at ‘O' level in fact. In 
these circumstances, the teacher’s counsel is one of caution and adventurous 
ideas may l>c sacrificed to pedestrian, though correct, writing. What could be 
an interesting exercise for the pupil turns out to be a very dull affair indeed. 
I doubt whether the quality of much essay work at ‘A’ level is such as to con- 
sider the essay a serious alternative to prose eonijx)sition. I should like to see 
this vaguely conceived exercise replaced by clearly defined areas or limited 
situations, where the terms of reference are laid down, where loundaries of 
language are picseiil>cd. The A EH shows some concern for this problem of 
limitation of linguistic held. The regulations governing the new ‘A* level syllabus 
state that ‘the linguistic fields covered by the essay subjects will have some 
tearing on the areas of knowledge contained in the topics or the prescribed 
texts of Paper IIP. This is a welcome move away from those essay topics that 
arc too fanciful or sophisticated, as, e g. : 

‘Vivir ci convivir o no ci nadjf (Jorge Guillen) 

La? vrnlajas dc la liuvia 
F.l valor dc la ainiiud 
hr* bicnfaili dc la pauvrcl^ 

‘La temperance et Ic travail v^.it It* dtu\ \rai» mrdrein* dc Fhotmi.c’ 

Lei dangeri dc la routine 
Waru m lachcn wir 
Frcundscliaft 
Tiaum und Phanlasic 

/ itfraturc jCii itiwlion 

A teacher is probably at his best when he is teaching what he enjoys 
and what he ljelicves in. It is in Pa]>cr III where he will find this freedom to 
oj>eratc and it is rielu that this should be so. With the laudable intention of 
getting away from the sv Hahns traditionally dominated hy 'classical* literature, 
the AEB has introduced a 'civilisation’ topic coupled with twentieth century 
texts, I can appreciate the reasons that prompt this innovation : much valuable 
time is lost in the process of interpreting classical vocabulary and s)ntax; read- 
ing contemporary and non-litcrary text* could have a beneficial cfTect upon 
the pupil's work in the foreign language; in short, a syllabus of this nature has 
more relevance. Nevertheless, I view this development with some concern, for 
it has merely exchanged one set of limitations for another, What seems to me 
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a much better arrangement, and one which will be to the benefit of sixth form 
courses, is the introduction in 1972 by the JMB of an alternative Paper III in 
French dealing with twentieth century texts, civilisation and a dissertation 
prepared in advance. Brief details of prescribed works are as follows : 

‘Candidates will be required to answer tlwcc questions, one question from 

each of Section I and 2 together with one other 10 be chosen from either 

Section J or Section 3. They will aho be required to submit a dbscilftlion 

in English on a topic of their own choice.' 

Section l 

1. Camus, I’Etranger 

2. L)h6tel, l'I)e aux oiseaux de fer 

3. Duhamcl. Confession de ininuit 

4. Ionesco, l*c roi sc meurt 

5. Prtvert. La pluie ct Ic beau temps 

$. Saint Exup^ry, Vol de nuit 

Section 2 

7. Cousteau, Le monde du silence 

8. de Gaulle, Mer.voiic* de guerre, Vol. JM (excluding l a Vie<o;rc and 
document*) 

9. J. Guyard. Ix* miracle fran<;ai» 

10. Mender France, Pour unc ilpubliquc moderne, Collection Idf'cs, 
Gallimard 

11. Sal ac row, Boulcvaid llurand 

12. Weil, La condition ouvririe, Collection td'is, Gallimard 

Section 3 

13. Molkre, \a misanthrope 

14. Balrac, Eugenie Grander 

15. Anatole France, Lei dieux ont soif 

I understand from the JMB that during tire preparation of the syllabus 
over the past few years, there has been a good resj>onse from teachers’ organisa- 
tions for suggestions of texts which would l>c appropriate to this new pa] *7 , 
The Ixmdon board, too, is developing a pajxrr similar in character. 'I his, I am 
sure, is the right direction in which to lx* moving : towards w idening the see]* 
of the examination, thus giving the teacher the freedom to choose the syllabus 
most suitable to himself and the conditions under which lie lias to work. 

What seems to roc to lr the greatest problem with Paper III is the 
paucity of reward in relation to the amount of work clone. Many teachers feel 
that the sheer toil involved ought to merit a much higher return. At the moment, 
preparation for Pajxr III takes cm the nature of cramming with all that this 
implies in the use of bad teaching techniques, 

Oral tfsi 

Ideally it is very desirable to increase the jKreentage awarded to this 
test. This would, ro doubt, have a jKwilivc. t*enrficial effect on leaching. But 
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u mil a test is devised that is susceptible of greater control than at present, merely 
increasing the proportion of oral marks would make nonsense of \V level 
gradings, Criteria of linguistic achievement — what :s desirable as a level of 
performance and in what area of language — are at the moment so vague that 
I am content to accept the existing mark allocation. What 1 should like to sec 
is the oral lest standing in its own right, divorced from the written examination 
and given its own grading. 

Postscript 

If W level is a leaving examination, then it is right that it should 
emphasise linguistic proficiency. On the other hand, universities (the long- 
established ones, at least) look for evidence of powers of literary appreciation. 
Whatever the role one a*cr : l>es to ‘A’ level, the fact is that at the moment it 
cannot satisfactorily accommodate the various groups, with their differing back- 
grounds and inclinations, which arc now a fc.iluir of our sixth forms. Some 
pupils have to l>o complete!) ki* om of examination reckoning; others spend 
two years dcsfrcralrly trsfi g to r t n with a scheme of work anil never really 
mastering it, lxcausc c nn*>:u w„ 'hu htniight alxmt the bV level examination 
have now changed and Ixx.v- , ir m> \icw the nrost cogent reason, insufficient 
thought has Item gi\e n to Mmi"* th\u1\ the area in which a pupil is asked 
to display his proficie r>< * 

In the ahseiv/ of ,t # U >r definition of objectives in linguistic terms, 
measuring proficirn'.j . i A' :*\c ! is very difficult, Performance is often implicitly 
compared with the m. ne akcr. ) 1 1 only a small nuinlxr of those who study 
a foreign latigu.ua nliinc an) thing hkr the equivalent mastery in all the 
areas of language avad tnle to the native sj>cakcr. 

In an effort to eater for the differing sixth form gioup«. it will probably 
l>c neo'svary to disidr the ‘A’ k vrl examination into self contained sectors or 
items. A candidate could then oiler a combination of these appropriate to his 
attainment, inclination ai>d require mcnls. thing existing methods of assessment 
would no doubt j**c problems, but a combination of ijkhIcs i and 5 might otTcr 
a satisfactory solution. 

Ike drawback to learning a foreign language and Icing tested in it. 
using traditioi al methods, is that the pupil is all the time sandwiched lelwcen 
huge slices of English and progress cannot always It* seen to l>e made. I he 
sixth form is a suitable place to rxjcnmcnt with block or intrmivc |criod< of 
teaching, particularly when a new language is begun from scratch. Hut that is 
quite another problem. 
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*A* level French: recent syllabuses 
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BOARD LANGUAGE LITERATURE ORAL 



AEB 



I Prose 5c Essay (31.25%) 
I Translation plus 
I Summary in English of 
a longer passage of 
French (25%) 



A: 3 prescribed topic i 
of ’Civilisation' 

B: 7 (20 texts with some 
social significance 
and loosely related 
to section A) 



Oral Re. ling 
5c 

Conversation 
(no Dictation) 

( 1875 %) 



5 questions: 1 from A & 
4 from B 

or 

2 from A 5c 

3 from B 

(25%) 



Cambridge 1 Prose 5c Essay (27%) 
2 Translation* (27%) 



A: 17, 18, 19, 20th century 
texts for detailed 
study. 

B : 18, J9, 20th century 
texts for less detail'd 
study. 

C: History, Institutions, 
Civilisation. 



I)ii lation, 
Reading & 
Conversation 



(19%) 



2 



questions from A and 2 
from any other section. 

( 27 %) 



J MB 



1 Trose 5c F.s*ay (32.5%) 

2 Translations (27.5%) 



A: 17, 18, 19, 20th century 
texts for detailed 
study. 

B: 20th century texts 
for less detailed 
study. 



Dictation, 
Reading 5c 
Conversation 



05%) 



4 



[urstions from A c-r 
from A plus 2 from B. 

(25%) 



Ix-ndon 



2 Proses A I Fx«ay (33 3%) 
2 Translations (16.7%) 



A: 17, 1 8th century' texts. 
B: 19th century- texts. 

C: 19, 20th century text*. 



5 questions, at least one 
from each section. 

(33 3%) 



Dictation, 
Reading & 
Conversation 



(167%) 
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BOARD 



LANGUAGE 



LITERATURE 



ORA! 



Oxford 


1 Prose At Essay (29.5%) 

2 Translations (23.5%) 


A: Voltaire (1969) 

Maupassant (1970) -f 
20th century texts. 

B: 17, i8, 19, 20th century 
texts. 

C: Genera] reading based 
on A &; B boo la. 


Dictation, 
Reading 4c 
Conversation 
(17.6%) 






4 question* — I from A, 

1 from B, and 2 from 
A, Bor G 

(29.5%) 




O 4t C 


1 Prose At Fas ay (28.6% ) 

2 Translation* (28.6%) 


12 texts, 17, 18, 19, 20th 
century, 

4 comment arid from 4 
diffcient texts plus 
2 essays from 2 
different set books. 

(28,0%) 


Dictation, 
Reading 4c 
Con vc nation 
(143%) 


SUjB 


I Prose At Essay (33.3%) 


A: I^i Fontaine plus 
Moli2rc or Racine 
( 1 909) 

B: 19, 20th century texts. 

All for detailed study. 

C: i 7, 19, 20th century texts 
for less detailed study. 


Dictation, 
Reading 4c 
Conversation 
(20%) 






0 questions: 1 compulsory 
and ] other in A f 2 from 
B and 2 from C. 

(30%) 




WJEC 


1 Trose Ac Essay (33 3%) 

2 Translations (16.7%) 


A: 17, 18. 19, 20th century- 
texts for detailed 
study. 

B: 2 books for hack jocund 
study of Civilisation. 


Dictation, 
Reading Ac 
Conversation 
(16.7%) 






5 qurstiens: all from A 
c* 4 from A and 1 from 

B. 

(»»%) 
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CHAPTER 2 



CSE Mode III examinations 
in modern languages 



A. P. DYSON 

‘EfTcctive teacher coi trol of syllabus content, examination papers and 
examining techniques is the rock on liicl j the CSE system will stand.* I his 
was one of the basic principles outlined in Examinations Bulletin 1\ By their 
constitutions the CSE examining boards are obliged to offer a range of examina- 
tion facilities to suit the particular needs of schools in their area so that this 
teacher control can indeed be effective. Ihcse needs have been catered for until 
now by the provision of three modes of examining. 

In Mode I, the boards publish syllabuses devised by tbeir subject panels 
of teachers, provide examination pafiers drawn up under the supervision of 
these panels, and mark and grade scripts again under the su|>emsion of subject 
panels and grading committees whose mcinlKTS are teachers. 

In Mode II, schools devi^ their own syllabus and the boards set and 
mark the examination, as in Mode I. 

In Mode 111, schools devise their own syllabus, a^ss the work of their 
pupils and suggest grades, in collaboration with a moderator rcsjronsiblc to the 
boards through their subject panels. 

In view of the general principle quoted alrove, one would think that 
Mode III was the technique ino^t suited to CSE examining, in that hr.e teacher 
control is at a maximum. It is rather surprising to Irani that ‘ I he Mode 111 
CSE examinations are taken by under 10 |>ct cent of CSE school* ar the 
moment* 1 . In the case of modern languages the percentage is even smaller. As 
will be seen from the table below, only three hoard*. WeU Yorkshire and Lind- 
sey, East Midlands and East Anglian, make u*c of Mode III techniques to any 
considerable extent. 

* The Cttttfucle of SefOndutt Edwtttion, Secondary $< I; cob Txarnir-.iiicn* <<un<il; 

F,ximin»tiorw Bulletin 1. IIMSO, l%3 

1 I. M. Conniughton, The validity of ex»minationi it 16 plus*. See Appendix 2 



*5 



This situation was forecast in Examinations Bulletin V; . . in some 
subjects there is a wide measure of agreement amongst teachers on the ground 
to be covered by the end of a five year secondary school course, and the fusi- 
bility exists of devising examinations that can test the standard achieved, ii re- 
spective of the different groups of pupils , . . / The Bulletin goes on to cite 
modern Languages as an example of ibis type of subject. 

When the Manchester University School of Education GSE research 
project began work in 19G3, it vvas noticeable that the Modern Languages Panel 
was able to agree, on general principles, on what it wanted to test in the new 
examination more quickly than its colleagues in the Art, English and Mathe- 
matics Panels. 

It is easy to see that Mode III techniques offer teachers great scope in 
some subjects, e.g. History, Geography, Art, Handicrafts especially. At first sight 
one might feel that since a language syllabus is more or less defined by the 
phonetic, lexical and structural content of the language itself, there is not a 
great deal of scope for variations in syllabuses which outline the attainment 
exacted of pupil s in their first five years of language study. In view of the 
problems inevitably jxased by Mode III examining, diseased at length in 
Examinations Bulletin 5 1 , one might conclude that Mode III syllabuses arc 
hardly justified in modern languages. 'Ilicrc arc, however, a number of grounds 
on which schools might rea^mably feel that a syllabus different from their local 
lx>ard's Mode I syllabus would be letter suited to the needs of their pupils. (One 
must remember that schools cannot "shop a?ound‘ from lx>aid to l>oaid for GSE 
examinations as they can for GCE examinations.) 

A school might feel, for example, that it did not agree with die iclalhc 
imjx>itance allocated to the various language skills in the local Mode I ssllabus, 
or that it did not like the testing techniques used in the lioaid's examination 
(the form of the teslq for example, or the u<e of l<i|>c recorders). 

Again a school might wish to include in its syllabus elements which did 
not form part of the Mode I tylhbu*. for example : background studies, project 
week, or set books. And a school might wish to use continuous assessment in 
grading its candidates. 

A further ca<c where Mode II! examining might l«e appropriate i« where 
a whool feels that the Mode I syllabus is not suited to the level of attainment 
it could ex|>c<t of its pupih. A school is free to submit a scheme of work which 
is notably less exacting than it« board’s Mode I syllabus. 

' P 5 
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This is a cmc wlirrc Mode i\i examining has great potential. Syllabuses 
could be deviled which would restrict demands on candidates to what they can 
cope with successfully : intensively, as regards range of tenses, vocabulary, struc- 
ture studies; or extensively, as regards the range of language skills handled. 
Anyone who has marked CSK examination scripts will know that a substantial 
proportion of less able candidates perform so badly in free composition that 
one wonders if they have gained much from their years of practice of this skill. 
Perhaps it would have been better in many of these eases if the pupils had fol- 
lowed syllabuses in which this particular exercise had been replaced by some- 
thing less demanding. Mode HI syllabuses could be devised which would involve 
the use of only the oral/aural skills or of only the listening/reading skills. 

Naturally, where syllabuses arc restricted in this way there will be a 
restriction of the range of grades which boards will award to describe the per- 
formance of candidates entered by the schools concerned. It seems a 
letter principle to grade candidates by limiting the scoj>c of examinations to 
material which they can rope with comfortably and grading them according 
to these limits than to set a comprehensive examination which will grade can- 
didates by the extent to which they are unsuccessful in coping with the 
examination as a whole. 

It may l>c argued that this function of grading the examination rather 
than the candidate could lx* exerei^d more efficiently through Mode 1 pro- 
cedures. An examination could be devised which would consist of a number 
of uni ts of graded d Ticulty : candidates would receive their examination grade 
according to the number of unit* attempted in which they had attained a satis- 
factory level. There would \*c a r cut off joint in each unit, with no attempt 
to grade within each unit. I hi* would make the lad; of the examiner 
easier; it i< difficult to devise test material in modern languages which dis- 
criminates evenly over the wide range of ability represented by the five CSK 
grades. But though this unit system might be theoretically jXKsihle in a Mode 1 
examination, there are obviously considerable practical difficulties. I he idea 
of a unit t>j>e examination in modem languages was lx mg di'cmscd in the 
early thirties when the Modern Language Association set up a research project 
to devise a translation. free examination, but no workable scheme for such a 
unit type examination was evolved 1 . It seems then that the idea of using graded 
test material can Ixst l«c exploited through Mode HI tcchiiupics and one would 
I ke to see more whools using Mode 111 facilities for this purpose. 

* In SiodfTn t nn^Mactf, sol. 14 nos. 3 and 4. February 1933 and sol. J3 rK-s. 3 and 4 
February 1934 
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A nothci situation in which Mode III techniques could lx? used to great 
advantage is where a school, or more likely a group of schools, wish to offer 
a syllabus which is tailored to suit a published course. Any school is free to offer 
such a syllabus but the first attempt to exploit the potentialities of Mode III 
in this rcsjKCl on a group basis is being made by members of the Nuffield/ 
Schools Council Modern Languages Project at York. 

The organisation of schools on a group basis appears to lx? the ideal 
method of exploiting Mode III techniques, as is pointed out in Examinations 
Bulletin 5 l . One can see, however, that boards might be suspicious of anything 
which apf>earcd to undermine their regional autonomy. They might feel that a 
Mode III syllabus organised on a national basis could do this. It is to be hoped 
that this difficulty can be surmounted. Such group Mode III examinations, 
organised under expert guidance, could provide an object lesson in the use of 
this under-exploited field of GSE work. On a national basis they might prove 
a useful point of common reference for the boards in ensuring parity of grading 
standards. 



Summing up 

We have seen that schools offering modern languages at CSE level make 
very little use of Mode III facilities. Yet Mode III gives schools the opfoitunity 
to design examinations to suit their own pupils, in terms of content and diffi- 
culty, and this latter possibility is paiticularly relevant when one considers that 
the ability range of pupils entered for CSE in modern languages is likely to 
be extended at the lower end of the scale in the next few years. It is to be hojxrd 
that the CSE boards will do more to extend the scope of Mode III examining 
by encouraging teacher* to use the facilities that exist, by streamlining modera- 
tion procedures and by welcoming group Mode 111 schemes when they arc 
proposed. 

See .Yofc ot ftlra/. 
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NOTE 



Table showing percentage of candidates in CSE Mode III French, 1969 

Xante of board Total candidates in Percentage in 

French (all modes) Mode III 



W. Yorks arid Lindsey 

E. Midland 

East Anglian 

North Regional 

Middlesex 

S. Western 

S. Regional 
S,E. Regional ... 
Associated Lancs Schools 

N. Western 

Yorks Regional 
Welsh Joint Board ... 
W. Midland ... 
Metropolitan ... 



3,267 


38 


3,344 


23.8 


5,234 


21 


2,414 


7 


2,297 


1.4 


4,461 


1 


4,049 


less tl 


3,654 


0.1 


1,343 


None 


4,842 


n 


2,349 


»» 


1,350 


ii 


5,158 


h 


4,149 


ii 




CHAPTER 3 



Non-school examinations 



J. B. ADAM 

There exist* outside the schools examinations system. examinations in 
languages set and administered hy no fewer than five nationally known oiganUa- 
lions — the Institute of Linguists, the Ixmdon Chamber of Commerce, the Royal 
Society of Arts and Pitmans. In addition, the Joint Committee for National 
Awards in Business Studies provides for examinations in languages as part of 
its Ordinary and Higher National Diplomas in Business Studies. 

It is, of course, true that very many language students who have left 
school and are continuing their education, either full-time or part-time, at col- 
leges of further education, evening institutes, etc., also direct their studs towards 
'O' level and ‘A’ level (ICE, with the aim of obtaining the necessary passes to 
proceed to a university, |>olytcehnic. technical college or college of education. 
On the other hand, there are very many more whose interest in obtaining a 
language qualification is determined by occupational needs. 

It is for the latter class of student that the examining l**Lrs mention'd 
eater. Whereas school examinations tend to me material taken from more or 
less literary sources as the * asis for examination questions, the non-school 
examinations in most cases make quite explicit that they seek (heir question 
material in 'economic, political, social, commercial and general scientific (topics) 
as found in good newspapers or journals' as well as in the 'geography. life and 
institutions of the country' (Institute of Linguists), The I.C'C in its wiitten 
examinations relics on 'everyday life', 'everyday topics', 'commercial corre- 
sj>ondcncc', 'commerce, industry and economics’, excluding highly socialised 
and technical vocabulary. "I he ChamUr a|s<> organises an oral examination: 
Torcign Languages for Industry and Commerce*. I he RSA emphasises 'the 
institutions, way of life and current affairs in the country concerned’ as well 
as matters of a ‘general character 1>cafing on industry and commerce or current 
affairs of the country concerned*. Language p.i|*rs in the Ordinary and Higher 
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Diploma* in Business Studies have, of course, a strong bias towards economic, 
commercial and industrial content but also favour social problems and current 
affairs. Only Pitman’s, of all the non-school examining bodies, makes no explicit 
statement as to the basis of its examination papers, but perhaps this may be 
taken for granted in view of tfie well-known association of Pitman’s with 
commerce. 

The recent, thorough-going review and revision of its examinations 
carried out for the Institute of Linguists provides a first class statement of the 
objectives and methods of approach which should characterise all non-school 
language examinations and which will undoubtedly extend their influence to 
the examinations of other bodies. It is made clear that the Institute’s examina- 
tions arc not intended to test knowledge about a language, or how' much of a 
given syllabus a candidate has assimilated. 1 hey arc proficiency tests— tests of 
the degree to which candidates can demonstrate their ability to apply their 
knowledge by ]>erforining certain tasks of verbal communication so devised as 
to l>c as realistic as j>ossib1c. This particular approach is followed most closely 
by the oral examinations of the LCC (Foreign Languages for Industry and 
Commerce) and by the RSA (Certificate for Secretarial Linguists and Diploma 
for Hi-lingual Secretaries) as will lie evident from the brief summary below. 

Although the oral approach to language teaching lias made considerable 
headway in the schools in recent years, this a<f>cct lias not yet received the 
weighting in (ICE at either ‘O' level or ’A* level which many would like to sec. 
Already in the Institute oi Linguists* various level*. in the RS.Vs Secretarial 
Linguist examinations and perhaps most of all in the LCG's 'Foreign Languages 
for Industry and Commerce’ the oral test receives the emphasis it deceives. 
Here perhaps the single subject examinations of the LCC, RSA, Pitman’s and to 
a great extent also the language pajxis of the National Diploma in Business 
Studies, lag somewhat behind, although the latter docs alL>l 40 per cent of the 
total marks to the oral section. 

!t may be useful to take each of the examinations in turn and to indicate 
briefly sj>ecial characteristics which distinguish them from others. At the same 
time, some estimate of comparative standard* at dilTcrcnt stages, where such 
a comparison i* possible, may l>c of interest. “1 be various stages of the Institute 
of Linguists and of GCK may *civc as standard* of reference of linguistic 
difficulty only. 



institutr of 

1 he Preliminary Certificate tests elementary ability to understand the 



spoken language and to speak it in conversation, to take part in situations and 
to show some general knowledge of the country. The Grade I Certificate in 
addition tests ability to read aloud, to write from dictation, to understand a 
written text, to write a translation into English of a foreign text and a free 
com[>ositiom Its linguistic standard is approximately equivalent to that of 'O' 
level. The Grade II Certificate, approaching 'A 1 level in linguistic difficulty, 
in addition to written translation both ways and a coinj>osition of 250 words 
on background subjects of the country concerned, lias oral requirements of a 
prepared talk, conversation, written summary of foreign speech and sight trans- 
lation both ways. The Intermediate Diploma is to be attempted only by pros- 
|>ectivc professional or semi-professional linguists and approaches the linguistic 
difficulty of a pass degree, while being more practically orientated. 'Ibis 
examination introduces, among other tilings, ad hoc interpreting, i.e. a test of 
ability to act as an 'interpreter in a conversation on everyday business or social 
matters lictween an English person and a speaker of the foreign language’. An 
additional, optional, sjiccial subject paper is available involving the answering 
of two questions from five relating to commercial practice, transport and 
insurance, banking, librarianship, travel and tourism or other approved sub- 
ject*. An oral test is also included. The final Diploma is linguistically of honours 
degree standard and includes consecutive interpreting and the choice between 
an essay of about 1,500 words, specialised technical translation or general 
interpreting. 

Motional Diplomat in Business Studies: 

This is a grou|ied subject examination in which it is fiossiblc for one, or 
at most two of the five pa fieri to be taken in a foreign language, Since at both 
the Ordinary Ixvel (approximately equivalent to Grade II of the Institute of 
Linguists) and at Higher Level (approximately equivalent to the Institute of 
Linguists* Intermediate) these arc the only language examinations in the non- 
school group which arc not centrally set and marked, a unified pattern is diffi- 
cult to discern, as might be cxjx'ctcd. 

Language papers arc set by internal examiners in each college and are 
approved by assessors. Scripts arc marked by internal examiners and reviewed 
by the assessors whose main ta*k is to maintain reasonably uniform standards 
throughout the country. In spite of this apparent freedom to innovate and 
experiment, however, mo*t examination paper? take the form of written trans- 
lation lr»th ways and a coni|K>sition. The oral test generally requires a written 
summary of a passage read aloud by the oral examiner, a dictation and a con- 
versation with the otal examiner. The oral test, which counts for 40 f*r cent 
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of total marks, is administered by external examiners selected by the colleges 
and approved by the Joint Committee. Their competence and standards appear 
to vary considerably. 

One noteworthy aspect of the business subject papers of the National 
Diplomas in Business Studies is that agreements have been reached between 
the Joint Committee and the main professional organisations such as the Char- 
tered Institutes of Secretaries, of Cost and Works Accountants arid others, 
whereby subject for subject exemption is granted to candidates who reach a 
certain standard in the National Diploma examinations. In view of the con- 
siderable progress being made by the Institute of Linguists towards becoming 
a fully professional body, the question arises whether it might not be advan- 
tageous to consider some similar form of exemption, in whole or in pait, from 
the Institute's examinations to be granted to successful candidates taking lan- 
guages in the Ordinary and Higher National Diploma examinations. 

The t.ondon Chamber of Commerce: 

The single subject examinations in languages arc unremarkable but this 
is not true of the Chamber's ‘Foreign Languages fo r Industry and Commerce*. 
Originally produced by the Birmingham Chamber of Commerce in rcsjromc 
to demands from industry, particularly in the Midlands, that a new oral 
examination was required, designed to lest the ability of employees to carry 
out tasks in the spoken language at various lex els with a strong emphasis on 
practical fluency, this examination is a departure in several ways from the 
traditional and is more akin to the oral sections of the Institute of Linguists* 
examinations. Pictures and photographs, previously prepared topics of |>crsonal 
interest, situations, recorded conversations and sight translations into English 
arc used, as well as discussion in the language of the candidate's social exper- 
tise. In short, the examinations 'arc designed to discover how much )ou know 
and not how much you do not know*. 

Th • Royal Society of Art 

Like those of the LCC, the RS.Vs single subject language examinations 
have few innovatory features. On the other hand, the new examination? for the 
Certificate for Secretarial Linguists and the Diploma for Bi lingual Secretaries 
recently launched by the RSA, and corresponding approximately to the Inter- 
mediate and Final standard* of the Institute of Linguists, show a bias toward* 
economic and commercial material. As lefils good secretaries candidates arc 
tested on their ability to compose letters and reports when given factual details, 
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to take down in and transcribe from foreign shorthand, to type letUrs ready 
for signature, to perform liaison interpreting between two businessmen. These 
are examinations of a high standard. On the other hand, many colleges already 
set internal examinations for bi-lingua) secretaries and it is understood that the 
Institute of Linguists is now considering offering examinations along similar 
lines. 

On the assumption that language examinations should test the eandi* 
date’s ability to perform a practical task of communication in the foreign lan- 
guage, it would seem that an examination devised somewhat on the following 
lines might be appropriate for non-school candidates : 

Oral 

Summarise a foreign text in £ngfisfi. This skill might well be 
required of an employee in business and commerce in supplying 
the gist of short business letters, rcj>orts from abroad, newspaper 
articles or advertising copy. 

Interpret a conversation betueen on English and a foreign 
speaker. Not all employers have retained anything from an ‘O 
level of twenty )cars ago. 

Translate at sight a foreign text into English, c.g. short letters, 
rejKJrls and articles for the cmplo)cr in a hurry. 

Converse in a foreign language tufJi one person. As when iccciv- 
ing foreign visitors, taking telephone calls, making business trips 
abroad. 

Converse in a practical situation. Application obvious. 

ITri/fen 

Summarise speech in a foreign language into English. Reaming 
the ability to give the gist in writing of sj>ccchcs at conferences, 
business meetings, to make notes of longer telephone calls. 
Summarise a foreign text into English. A test of ability to give 
the got of a longer letter, rcjoit or newspaper ailiclc. 

Translate a foreign text into English. A test of ability to give 
an employer a translation of a letter, report, contract, or atticlc 
from a periodical. 

Original composition in a foreign language. To test ability to 
write a |)CfSonAl letter or short business acknowledgements, 

finally, can such a proliferation of different bodies responsible for non- 
school language examination* bo justified? Would there not 1>e economies to be 
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gained from rationalisation? The equivalence of standards of the various 
examinations is by no means as close as the rough approximations attempted in 
this [taper would suggest Would there not lx* much to be gained by standardisa- 
tion under one or two t>odie$? 'l he prospective employer has a fair idea of what 
‘O' level or ‘A* level means. Hut what is to be made of Grade I Institute of 
Linguists, Intermediate LCG, lntei mediate USA, Intermediate Pitmans, to say 
nothing of Ordinary National Diploma in Business Studies? It is about lime the 
non-school examinations l>odics got together. 



NO IT, : I am indebted to Mr P. J. LoeVe *nd 7 ht Incorporated Linguist for suggesting 
the bam for the following comparative table which I have modified and extended 
to cover non school examinations bodies 
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COMPARATIVE TABLE 



Oral 

1. Summarise in French a speech in French 

2. Summarise in English a text in French 

3. Consecutively interpret in English a speech in French 

4. Interpret a conversation between English and French speakers 

5. Read a French text aloud 

6. Translate at sight into English a French text 

7. Translate at sight into French an English text 

8. Converse in French with one person 

9. Converse in French in a practical situation 

10. Make a prepared speech in French (with/without notes) 

11. Make an unprepared speech in French 

Written 

12. Answer questions in English on a speech in French 

13. Answer questions in French on a speech in French 

14. Answer questions in English on a text in French 

15. Answer questions in French on a text in English 

IG. Answer questions in French on a text in French 

17. Summarise in English a speech in French 

18. Summarise in French a speech in French 

19. Summarise in English a text in French 

20. Summarise in French a text in English 

21. Write from dictation 

22. Translate into English a text in French 

23. Translate into French a text in English 

24. Demonstrate knowledge of the country 

25. Write original composition in French on given theme, 
or in response to text or speech 
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CHAPTER 4 



Ora! examinations 



E. F. CHAPEEN 

I pro|> 05 -c to discuss two reasons why it might be preferable to obtain 
measurements of oral proficiency from the teachers of those being examined 
rather than by means of formal tests conducted by external examiners. I will 
concern myself mainly with the reliability of w hat are by now tiaditional methods 
of oral examination, and with their validity. 

r l1ie traditional methods of oral examining boi row something of ‘country- 
hou<e’ methods of selecting civil servants and seivice officers, and something of 
the methods of fonn.il speech training and examining. Thus, the candidate may 
have to give a short talk on a lopic presented to him ten or twenty minutes l*cfore 
to a small group of fellow examinees, then answer questions put to him by 
members of the group --possibly under the prompting of the examiner. Or hr 
may have to di«cus$ a visual stimulus with the examiner a map. a painting, or 
a town plan, (or example. Or he may have to read a set piece either to the 
examiner alone, or to a small group of fellow examinees. But whatever the exact 
form the test ta<k takes, the examinee is placed in a situation which for the 
purjKtsc of standardising and comparing performances is a* highly controlled „s 
possible, ITcn his pci foi inane e is assessed on a closely defined scale of ability 
by one or more examiners. 

I rom one |>oint of view this appears to l>o a highly satisf.ictoiy type of 
examination : the candidate is being required to demonstrate the ability that 
most |xx>p!c would agree i* exactly the one we widi to measure his ability t« 
communicate orally. Unfortunately, however, no matter how clcwclv defined i< 
the scale of ability against which bis perfoimancc i< rnc,vwcd. the measurement 
is made subjectively and is, in consequence, unreliable. Of course, this is rot a 
sufficient rcAvan in itself to condemn the examination. Indeed, (be only complete 
certainty in psychometrics is that any measurement we obtain is in eiror. How- 
ever, if the amount of error involved in oial examining i« great that little 
reliance can 1c placed on the neat majority of decisions that one makes on the 
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basis of the examination results, there is little |>oint in going to the expense and 
trouble of constructing and administering such tests. 

Hie crucial point in the mark distribution of any examination is at tbc 
cut between pass and fail- If, as is very* frequently the rase, the cut is made at 
or about the mean, and if, as is also frequently the ease, the great majority 
of scores duster at or about this j>oint, and if, as is also frequently the case, 
the reliability of the examination does not exceed 0.050, then the likelihood is 
that about 40 per cent of the candidates' score's will fall within the area of gross 
error at or about the cutting point. That is to say, there is a 10 per cent or 
greater chance that the true score of each of the 20 per cent of candidates whose 
obtained scores cluster immediately above the mean falls below the mean, and 
a 10 per cent or greater chance that the true score of each of the 20 per cent 
of the candidates wh<h^ obtained scores cluster immediately below* the mean 
falls alovc it. 

In order to give some indication of the pro] >or lion of scores one might 
e\|HCt to fall within the area of extreme error about the cutting joint in the 
score distribution of an examination, two types of daia arc necessary; first, the 
score distributions themselves, and secondly, reliability coefficients for the grading 
procedure. In the rejort of the experiment in oral examining conducted by 
the Southern Regional Examinations lk>ard in 1964 1 , we can find everything 
\se need except the score distributions. However, the smallness of the standard 
deviations obtained in the experiment, and the fact that the total jossiblc range 
of marks was employed in only one of the four Mib-te^t*, strongly suggest that a 
large projortion of the candidates were placed in the area of gloss error alout 
the mean. 
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Reliability coefficients in oral examining taken from 
School i Council f laminations Bulletin 11 t p. 19 



METHOD 


Examiner 

V 

Observer 


Examiner 

V 

Assessor 


Examiner 
+ Observer 

V 

Assessor 


Observer 

V 

Assessor 


N 


Reading 


— 


— 


.804 


-- 


90 


1'opic 


.837 


/>74 


— 


.591 


89 


l isuat 


Stimulus 


.877 


.730 


-- 




99 


Croup 


Di reunion 


.785 


— 


— 


* — 


103 


1 Schools Council FuUclin II. See Appendix 2 







NOTES : 

For the reading method the examiner and observer produced an agreed mark. 
Consequently, no reliability coefficient is available. The two people concerned 
reported very close agreement and no difficulty in coming to an agreed mark. 
For the group discussion method it was impossible to assess from the tape 
recordings. 

Examiner : the person in charge of the method. 

Observer : the person taking no part in the experiment but sitting unobtrusively 
at the back of the room. 

Assessor : the person assessing from the tape recordings. 

TABLE II 

Means, standard deviations, and effectiie range oj the 





reliability data 


reported in 


Table / Bullet in 


//, />. 23 




M F T HOD 


Examiner 


MEAN 


S.D, 


RANGE 


N 


Reading 


+ Observer 


6.1 


1.87 


MO 


90 




Assessor 


5.5 


2.44 


MO 






Examiner 


6.3 


1.37 


3*9 




Topic 


Observer 


6.2 


1.41 


3*9 


89 




Assessor 


5.6 


2.39 


MO 






Examiner 


6.5 


1.47 


2*10 




1 iiual 


Observer 


6.4 


1.58 


3*10 


99 


Stimulus 


Assessor 


5.4 


2.28 


MO 






Examiner 


5.6 


1.96 


1*9 




Group 


Observer 


5,9 


2 29 


MO 


103 


Discussion 


Assessor 


— 


— 


-- 




Since it seemed unlikely that the 


score distribution* 


for the SR KB experi* 



rnent would be easily accessible, application was made to the Joint Matriculation 
Board for |>erniission to make use of the score sheets for the Optional Oral Paper 
in the Ccncral Studies (Advanced) examination administered in 1 969, and this 
the bo, ml \va« generous enough to giant. 

Unfortunately, the reliability data that txht for this administration of 
the JMH Oral 'lest arc based on samples tanging in si?c from seven candidates 
to thirteen, and it was felt that these were too small to justify even tentative 
conclusions. For tlii* reason it was decided to make use of the reliability data 
obtained during the SR KB cxj*rmvnt. Although the foinnof oral test u«ed by 
the SR KB in this experiment differ in certain respects from those employed by 
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the JMB, and although the two populations being tested were different, the 
task of the examiners was the same in each case : both teams of examiners had to 
rank the candidates on a tempoint scale of competence in oral communication. 
And it is the reliability with which this task can be carried out that we arc most 
interested in at this point. 

One further liberty was taken with the reliability data for the SREB 
experiment. In this experiment a different examiner administered each of the 
sub-tests to a different group of piupik It was therefore impossible to determine 
experimentally the reliability of the entire battery. In the JMB ‘1‘cst, on the 
other hand, we are in possession of the marks awarded to each candidate on 
each of two sub-tests, and it was felt that it would be of interest to sec how the 
increase of reliability that results from doubling the test length affected the 
proportion of candidates’ grades falling in the area of gross error alout the mean. 
It was decided, therefore, to hypothesise a total-test marker reliability of 0.850. 
'Hie full details of the reliability coefficients assumed for the sub-tests in the 
JMB Oral Test arc set out in Table III below. 

TABLE 111 

Assumed reliability coefficients (or the sub-tests in the 
Joint Matriculation Board's optional oral test in English 
METHOD RH IABIUTY COFFFICtFNT 

1. Talking to a given topic 0 837 

2. Conversation uith another examinee 

on a given topic 0.783 

3. Heading a set piece 0.884 

4. Total test (i.c. 1 ± 2 or 2 3) 0.850 

In all, two moderators and twelve examiners were Involved in administer* 
ing the JMB Oral Test to about 1,800 candidates. For the purpose of this 
investigation the score sheets of four examiners were selected, one of the 
examiners having had eleven years' experience with the test. o*«e having had 
more than five years' experience, one with one year's previous cx r i lienee. and 
one who was newly appointed in 1969. All. together with the other examiners 
concerned, had attended a full day's standardising meeting during which about 
a dozen pupils representative of th? sample of the population being examined 
were tested, and the performances and results discuss'd. 

On the basis of the assumed reliability coefficients set out in Table III. the 
standard error of each of the «iib-lcst$ marked by each of thr four examiners 
selected for this investigation was computed. Next, ogives of the relative 
cumulali\c frequencies for each sub-test marked by each examiner were con- 
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structcd. Then, assuming that the cut between pass and fail was made at the 
score point closest to the mean, the 20 per cent, the 10 j>cr cent, and the 2 per 
cent confidence limits of that score point were calculated, and the proportions of 
candidates falling within these limits read off from the ogives. 

'Hie re>uhs of this investigation of the mark distributions of the four JMB 
oral examiners on the basis of the assumed reliability coefficients set out in 
Table 111 show that if these coefficients are realistic estimates, between 36 per cent 
and 59 per cent of the candidates fell in the area of gross error about the score 
jx>int closest to the mean in the ease of a single test of ora) proficiency, and that 
in the case of a double test, between 36 per cent and 45 per cent did so. Further* 
more, if one works to the 5 j>er cent level of confidence, one can he reasonably 
certain of the pass/fail categorization of between 38 per cent and 54 per cent of 
the candidates in the case of a single test, and between 43 j>c r cent and 52 per 
cent in the case of the summed results of two sub-test*. 

The conclusions we have drawn from the results of this investigation arc 
necessarily tc itative. We have taken raw-score data from the sub-tests in one 
form of oral examination and to them applied reliability data from the sub-tests 
in another. In addition, we have assumed a reliability foi the entire test on 
the bad* of reasoning and not c\|>ci intent. Hut if it is agreed that the examiner- 
reliability coefficients which have been employed arc realistic estimate*, then we 
suggest that the conclusions we have drawn from this pilot investigation justify 
the setting up of a fully controlled experiment in order to confirm or disaffirm 
them, l or if one can he reasonably certain a!»out the pas* /fail categorisation on 
the bads of their [Kifoimnncc in an oral examination of only alout 50 j>er cent 
or fewer of the candidate*, there would !>c very little to lose and, in the opinion 
of many, a great deal to gain, from turning to teachers for assessments of oral 
proficiency rather than to external examiners. 

1 would like to empha*i*e that although this investigation has been based 
on the result* of the oral examination conducted by one particular examining 
l>onrd. the conclusions are by no means a reflection of the lack of comj>ctcncc 
of either the Ixurd or of the examiners concerned. As our examination of the 
standard deviations in Table It revealed, the same situation was probably present 
in the SKKH experiment : Mich considerable bunching of scores about the mean 
that c\cn a marker reliability of the order of 0.90 would only minimally relieve 
the situation. Furthermore, this state of affairs is to Ik* expected in any examina- 
tion that is subjectively marked and that consist* cszcntially of five or fewer 
items. 

So f?r 1 have been dnemdng only the reliability of marking a candidate's 
fKiformarice on a particular occadon. Howrxer, marker unreliability i* only 
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TABLE IV 

The percentage of candidates falling in the 20 / c , the 10 / 0 and the 2-/ 0 confi- 
dence limits of the score point nearest the mean mark obtained in the JStB 
optional oral paper, 1969 , assuming the reliability coefficients set out in Table 

III 



Exam* 

iner 


20° o 

Confidence 
Limits | 


10% | 
Confidence 
Limits j 

1 


03 ’ 

1 *■ O 

Confidence 

Limits 


i 

i 

N 

1 , 


Mean 


1 

s.u. 


Po&*. 

Range 


1. talking to a given Tone — Assumed reliability : 0.837 


A | 


j 39% 


W% 


65% 


40 | 6.06 


1 1.86 


1-10 


o 

1 


| 46% j 63% j 53 J 6.76 | 1.66 | 1-10 


2. CONVERSATION WITH ANOTHER IMMINKK ON A 

given topic — Assumed reliability : 0.785 


A | 42% | 55% 


74% 


| 108 5.26 


| 1.40 | 1-10 


B ; 53% | 65% 


«2% ! 


109 6.49 


1.70 


MO 


C : 


| 44% j 54% 


70% | 


147 j 6.18 


1.13 j 


1-10 


n 


| 46% | 57% | 


I 70% j 97 j 


j 6.65 


1.36 j 1-10 


3. keamno a set PIECE- Assumed reliability : 0.664 


A j 59% | 65% 


! 60% i 


108 ’ 5.80 


1.93 | 1-10 


B | 55% | 66% 


80% 


109 5.92 i 

1 ! 


! 142 


1-10 


c 


| 59% j 72% 


63% ! 


! 147 1 


6.11 


1.69 


1-10 


1) 


53% j 63% | 

i 1 


1 77% 


97 j 


! 6.47 

i 


1.34 


1-10 
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TABLE IV (continued) 
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one |krmMc source of euor. and ii U at least arguable that in the case of oral 
examining, it is a less important source than that of candidate/cxamincr or 
candidate /fellow-examinee interaction. As the SREB report states 1 . 

1 1 he m^st substantial obstacle to the conduct of a satisfactory (oral] examination 
is the attitude of the candidate* toward* the examiner. With unfamiliar and 
senium acquaintances, children tend to adopt a homogenous tone which is quiet 
and cotourlc's, and which rfler* a funny mixture of shxnrss and respect When it 
is compared with cU'snoorn experiettee it becomes clear that with tne best inten* 
lions imaginable, the examiner mav tra.il behind him a somewhat inhibiting 
atmosphere which remains an ireradicahle part of any conservation. Between the 
ace* of, say. twelve and seventeen, the children tend tn assume a guarded, some- 
time* resentful, reticent altitude toward* the adult world. This familiar social 
pO’blem can be intensified by the atmosphere of an oral examination. Such 
reticence, and perhaps an attendant nervousness, evrn awe, before an ur.kne>wn 
figure (however friendly) xcsted with influence ard authority, could easily prevent 
the essential flow of sympathy between examiner ard candidate, and the comcrsa- 
lion would be hailing and inept.’ 



The extent of error that this factor introduces into die measurement 
obtained from a test conducted by an external examiner can never be known, 
but we may suspect that it i< not inconsiderable. Certainly, its presence and effect 
lend strong supjxut to the argument that ill the case of oral examining the 
increase in validity to be gained from making use of teacher assessments instead 
of those made by an external examiner would more than conif>ciwatc for any 
loss in comparability of results from school to school that might arise as the 
consequence. 
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CHAPTER 5 

Research by examining boards 



H. S. OTJ'ER 

This must necessarily be a very brief survey of a very complex subject, 
and I shall refer only to the COE boards. I shall give first an account of some 
of the operational research which is a consistent built-in aspect of our woik and 
then of some of the specific post-examination research which is constantly taking 
place. 

Reliability 

Reliability is o'tcn too easily thought of as scorer reliability. Whiht this 
is very impoitant, the whole problem is exemplified very clearly by the broad 
field of the discussions at this conference, which base ranged from concern 
regarding the reliability of oral examiners— including self-coioistency and co- 
ordination problems— -to the extreme, where, with objectively scored tests we 
are concerned with the performance of t Tie test itself tcst/re-[est reliability 
etc.-- -with some comment by the way on the more conventional mark schemes 
associated with traditional written papers. In all these areas it is very easy to 
drift if not fall into the fallacy of relevant objectivity. We have to l>c wary of 
mere counting and arithmetical conveniences, llic tally counter introduced into 
j some of our earlier experiments in oral examining can be very useful, but one has 

: to lx* very sure of what one is counting and why. and that it is worthwhile. We 

i have to l>e wars’, in other words, of sujxrfieial reliability without validity. Reli- 

I ability must always lx sulwervicnt to validity. 

jj 

\ Comparability 

| l -cm par ability, I take it. is a convenient expression for some broader 

! aspects of reliability. It may lx thought of as starting when there are two 

examiners. In general, we use it with reference to the pass stand itds or grades in 
t :io or more examinations which are ojxrating at the same level in the same field 
and therefore may lx r ‘turned to lx compaiahle. e.g. the |»a«s in k O‘ hvei English 
Language, the Grade O/Bl-ordcr in 'A' level Italian. Ihrrc are. however. maro 
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aspects of this comparability and it P a very complex question : 

(1) h'rom year to year in the same subject in the same board , Some of 
the factors contributing to tills are the continuity of examiners and 
moderators, consistency of methods, availability of detailed statistics. 

(2) From one language to another in the same examination and the same 
board . In consideration of this problem the Cambridge Syndicate 
has taken a cross-reference analysis of the performance of candidates 
taking two language*. Inferences from such an analysis are based 
on the hypothetical quality of the candidate as a ltngubt and there 
•arc many other imj>ondc rabies, such as length and nature of course 
etc , but the figures for a large sample of candidates provide some 
useful information. 

(3/ The same language in the same examination for different boards. 
Investigations in this field, as I shall mention I. Her. are undertaken 
from time to time. 

It would be difficult to trace historically die developments of the collective 
conscioii'iiess of the l>o.irds; this was no doubt oi igmally very much a matter of 
agreement of like minds and in a relatively small world, an Academic and 
professional consensus. 

From 1919 the Secondary Schools Examinations Council and later its 
successor, the Schools Council, have had ginci.il oversight of the work of the 
examining boards. The Schools Council Subject Committee* have access to 
syllabuses, question papers, mark schemes and sciipt*, Comparability generally 
a; ‘/V level is the province of the Working Parly on Advanced bevel Compara- 
bility . 

Before 1939 the School Certificate Examination of each Ixvrd was scruti- 
nised [tcriodicallv by a team apjxvntcd by the SSEC. Sli ce then this has lapsed 
arid the Ixvuds have undertaken the icqomibilily of initiating acnniirming series 
of investigations into dilTrrcnt subject*. It i* necessary to mention the field 
as a whole in order to keep the matter in perspective : any one mr*dcin language 
is only one c»f four or five commoner languages examined and only one of some 
thirty subjects at \V level and forty or fifty at 'O’ Kvel. As I ref* tied to 'four 
or five commoner languages' I should undi d you that the problems do not stop 
there: at the Cambridge Syndicate we have recently Ixcn comparing the 
ipp.oach. style and star d ud of papers and examiner* in Norwegian. Turkish 
a i id Russian ) 

Each subject then mml take it* turn link < q vr . 1 on umstancc * inter- 
vene smb a*, for example, statistical anomahr i or q< « i'?c » title urn. Per modern 
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languages as for other subjects, detailed comparative statistics arc prepared each 
year showing the number of candidates and the results according to categories 
of candidates based chiefly on the types of schools. A great deal of information 
of this nature pros ides a continuous check on discrepancies and anomalies which 
may at any time be the subject ol further investigation. 

It is interesting to note that then* is general opinion, supported by 
research, that the problems of reliability in the marking of language papers are 
smaller than thoc of other subjects. 

In the course of these definitions I have given some notion of the concern 
of the boards and of their formal responsibilities and some of their method*. I 
should like briefly to remind >ou of some of the built-in processes directed towards 
achieving maximum reliability. These include the provision and comparison of 
mark schemes, elaborate processes of co-ordination, the study ot a w ide range of 
statistic** which involves the preparation and handling at each exainir alion of a 
\f -t quantity of material and records, all of which amount to a considerable 
research project and do in fact fill volumes. There arc also procedures of check- 
ing winch arc really a continuation and finalising of the co-ordination of mark- 
ing. Subsequently there is routine frost-examination analysis; details are usually 
rejoorted to subject committees and to the School Examinations Committees and 
in annual rej>orts on the examinations. Similarly, comparative statistics are 
studied by each board and discussed jointly where necessary. 

One useful factor for the processes of comparability may be the use of 
common element 48 . Ihcse exist Irctwccn scone l>oards in the use of common 
objective comprehension tests in some of the alternative syllabus* s which are 
now being taken by a large number of candidates. 

following a recent investigation conducted by a sub-committee of the 
Schools Council Modem Languages Committee further discussions between the 
l>oard$ have led to the cx|>crmicntal introduction of a common prose in the 
french examination at 'A* level in 1972. r |1>c Oxford Delegacy is co-ordinating 
the arrangements for this. 

Investigations are carried out from time to time by various Ixwds acting 
on l>ehalf of and in agreement with other hoard*. It mav be of interest to giv'* 
some account of the recent investigation organised by the JMB. on il e french 
*0‘ level examination in 19G9. Sample scripts from each Ixxird were marked hv 
examiners of all the other Ixxards in terms of their own standards and tlu f Chief 
Examiners reputed on the sy minor*, question pape rs. arid maik schemes of each 
of the other loaidv Ibis resulted in seventy-two separate reparts and a con- 
siderable aTi'Ount of statistical data. Smb an imrstigation aims at an assessment 
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of the total demands of the examinations and includes a great deal of judgement 
on matters not entirely susceptible of statistical evaluation. *Ilie real benefits of 
such enquiries can be summarised as follows : 

(1) Each board receives a critical assessment of all the aspects of its own 
examination. 

(2) Each board has samples of the content and method of other exami- 
nations for detailed consideration. 

(3) ‘Hie papers and meetings give an opportunity for discussion of the 
forms of the examination, notions of standards weighting of parts 
of the syllabus etc. The rcfxnt on tikis work will lx* circulated to all 
boards and will no doubt be s'nt to all their subject committees. 

Anyone concerned with the examinations is at present aware of two 
conflicting tendencies : the first, a proper concern about absolute reliability and 
comparability; the second, a strong and understandable desire for freedom in the 
form and content of syllabuses. The latter leads to experiment and diversity in 
methods of examining, and this is obviously necessary to provide c\j>ericr.ce and 
continuous improvement; though often, exponents of change tend to think 
that their method is the final solution. Hut whilst we must experiment we mu<t 
also realise that alternatives are bound to prejudice reliability. 

Other chapters of this rcj>ort mention the educational aspects of exami- 
nations. Although the subject of this paper is the reliability of the examinations, 
f mentioned at the outset that this cannot in practice be divorced from con- 
siderations of validity. Ilie boards and their adv isers, who are largely teachers 
directly involved with the examinations, arc constantly concerned with the 
educational implications of their work. But the boards cannot separate their 
rcsjx)nsibili.tics; they arc rightly cxjxrctcd to maintain reliability and continuity. 
*lhcy have to recognise genuine advances in educational inodes and functions. 
Ibey have to live with the problem of reconciling ideal reliability with educa- 
tional ideals. 
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The use and influence of objective tests 



W. K. rRKSCiO J 1 

Ati objective tot i* generally described as one in winch the questions 
are set in such a way as to have only one correct answer. The term ‘objective', 
therefore, reb'r.s to the marking piocrdurc only. Devidtig t lie test remains sub* 
jective. I he characteristic contribution of objectiv e testing has been that it 
has made |>o>siblc rapid reliable marking regardless of number.*. It follows from 
this that wherever such considerations arc imj)ortarit it is valuable at least to 
consider the feasibility of u«ing objective tots. 

The coKfhuction of objective tats 

Although objective tests are distinguishable from other forms of test by 
die cornplt to reliability of marking. a number of other characteristics have come 
to 1* associated with tlicir construction. These arc worth recording here since 
they have undoubtedly bad an influence on the formulation of objectives in 
language teaching and on testing generally. 

d) ’lhc major influence has I wen to shift interest away from an over* 
preoccupation with W4rJlrrr£ examinations to a consideration of the 
preparation of the examination itself, flic essential fust step in the 
construe tion of an objective test as indeed in the construction of 
all tests should l>c the detailed definition of objectives. Only "hen 
objectives have Iky n clearly defined can one hoj>c to disoiv* possible 
techniques of testing. Defining objectives in testable terms i«, of 
course, extremely difficult but it is the lack of just such a definition 
v. Inch makes discussion of ‘O’ h vt 1 language examination techniques 
so difhcult. 

id I hr definition of the teaching objectives is followed Iv detailed 
.inal\*is of the pnenc content of the trading progi amine. Ibis is 
particularly important in the e.isr of languages since only a hmited 
selection ftom the language can normally !«' taught in sdionls. arid 
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it is very imjxutant that the test constructor should have a reasonable 
indication of wlj.it this selection consists. 

Objective tests try to sample as widely as ]>ossib!c from the 
course content, and in this they often differ very markedly from 
trad tional examinations. 

(iii) The analysis of objectives and course content is followed by the 
w riting of lest items. With objective tests ihis is generally accepted 
as being a group procedure and a number of specialists are asked to 
contiibute questions. Ibis procedure, it might be suggested, lias a 
considerable advantage over lire normal procedure adopted for 
conventional examinations where on the whole far less material is 
considered at the preparatory stage. 

(iv) The try-out of the forms of the pre-test and the item analysis which 
follows go a long way towards ensuring that the test which is 
ultimately presented to pupils as a part of their examination will he 
reliable, will he of suitable difficulty, and w ill discriminate * dectivelv 
over the ability range. I he item analysis is. however, only one pait 
of the validation piocc**. and it is very iiii|*oit.int that the final form 
of the test should !>e carefully considered to ensure that it still 
reflet u. in a balanced way. the aim* of the couisc of instruction. 

It would be very sm pining if the effects of this careful 
preparation Were rot to be fell in other less objective parts of the 
examination. 

I tjfuf of mine obji't'tu r technitfurs nr ( * C.h. and CSE ( \ a initiations 

As mentioned catlir. before deciding on testing techniques. it is essential 
to clarify objective*, ft then lx comes appan nt whether objective techniques can 
be validly mcd. 1 ho increase >n rcliahjlitv cannot he achieved at the cx|»rn*c of 
the validity of the test. Hut presuming that objective tr*ts may he used, they do 
appear to offer a number of advantages at the f*C'K and CSK level*. 

(i) Large nuint>ors of pupils arc involved. 

(ii) It is desirable that the marking l»c completed as rapidly as |>ossil>le 
since many decisions tct)d to depend c*n the results. 

(iii) t here is a need for reliable tests which may help examining board* 
in the difficult l.vk of comparing pupils whnatc sitting similar exami- 
nations. In C'SK thue is the problem of comparing candidates 
following ditfrieiit mode*, and in (i(K the pie*hlnn of candidates 
following Alternative S\ 1lahu«rs or taking a similar examination w it h 
a different loaTd. With loth examinations there i« the problem of 
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comparing pupil performance from one year to the next, and also 
the problem of comparing standards in the two examinations 
(especially Grade l Candidate CSE/GCE 'O' level pass candidates). 

I he of object!: r tests in language testing 

In addition to the advantages that objective techniques may offer to 
testing generally, tire particular advantage they olFer to language testing is that 
they enable comprehension to be tested without involving the pupil"; ability 
to sj>eak or write. I bis is of very great impoi lance in tho^c ca*cs where the 
activities of listening and reading comprehension figure very prominently among 
the course objectives. 

(i) Testing comprehension 

A number of examining Ixxirds have introduced, or are considering intro- 
ducing. objective techniques of testing in the field of listening and reading 
comprehension. Although these tests might well be considered an advance on 
previous examination procedures they have mol with some criticism. It might be 
interesting to speculate on some of the reasons why some teachers are unhappy 
about their use *. 

(a) llie specimen material \\hich has been available has not always been 
of the highest quality- often rejected test items, Ibcrc is a tendency 
for teachers to be so affected by the quality of the items in the sj>eei- 
mcn material that they are not able tn appreciate the merits of the 
technique. 

(b) A number of the examples of multiple-choice tests which hav e been 
available have led teachers to Miqxxt that they involve .an excessive 
degree of intelligence over and a!>ovc a knowledge of the language. 

(c) Many teachers *arc still trot happy .about the |xwsibililic> of guessing 
This unease is often linked "ith a considerable degree of vagueness 
alx>m the significance of test scores, means standard deviations 
range etc. 

Id) ITierr is still a feeling of dissatisfaction with the listening compre- 
hension tests which have so far been dcvelnj>cd. Although these 
obviously involve a large measure of listening comprehension, tire 
reading element is s|j|I present to a significant degree. 

(c) Some teachers fear the |M>$ih!c ‘backwash* effects of multiple choice 
techniques. 

Ihc«c criticisms |x>int to the need for more discussion of the patticular 
technique? lacing used in ti e multiple-choice srttions of the ‘O' level examina- 
tions. 1 here is a danger that the particular tvpcsof item which were taken from 
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American tests may be used to excess. T he ‘Who is speaking?’ and ‘Where are 
they?’ tyjx's of item arc probably more suitable for a standardised proficiency 
test than for continual use in Min.ua! examinations. The situational techniques 
used in the listening comprehension test might well be reviewed to see whether 
or not another technique could Ik* found which involved less dcfiendcncc on 
comprehension of the written options. 

The question of intelligence is also an import ant one. With the agree- 
ment of the examining boards concerned, the Nuffield /Schools Council hrojcci 
has arranged for the pre-test of (!CK multiple-choice German and Spanish 
items on native speakers of the languages concerned. Classes of mixed ability 
of the appropriate age range have been asked to complete the tests to ensure 
that those items which appear to require qualities other than comprehension 
of the language arc readily identified. 

The likely backwadi effect of multiple-choice is still hard to assess. 
Vernon offered the view that 'So long as the objective questions arc rearonably 
straightforward and brief, we know that the amount of improvement brought 
about by coaching and practice is limited. . . . However, it is ]>ossiblc (though 
there is little direct evidence) that facility in coping with more complex items 
is more highly coachahlc and that pupils who receive practice at these may 
gain an undue advantage* 1 . One |>o>sih?c way of ensuring that the item l>pcs 
arc straightforward and lacking in complexity might W to adopt the procedure 
often recommended in the construction of psychological u*ts namely to con- 
struct the test first as an invent ivc-nqoi sc test and then to use the incoircct 
answers as a guide to f>ossihlc distiaclor*. 

(ii) Vesting written and oral production 

Some of the testing technique* often cinplo)cd in examinations to test 
speaking and writing arc of very doubtful reliability. Hie unstructured essay 
and the free conversation I >oth require subjective judgement* by examiners and 
arc very difficult to mark consistently. Writing and shaking, however, both 
involve qualities which it is inipo**ih1r to asses* entirely objectively. Ihc prob- 
lem is therefore how to reduce the unreliability of the marking without destroy- 
ing the validity of the test. 

fa) Ifrr/inj? 

The reliability of the l-M of writing may I*- improved in a 
number of w.ns ; bv increasing the tiuhiIkt ed questior «, increasing 

'Thf Cfrlificctf cf SftcrM&t} V.dMtoU:*: .4 n rsO -'furl*** f 7 t 1 6r>, : .r>cri -ft*, 

F.\aminitk‘ru Bulletin 4 . ScotxJary Seh^r.;* K\»rr.ir-*ttm (Vuncil. 1IMSO, 19C4 
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the number of markers so that a form of multiple marking is 
employed, or using analytic marking schemes. 

Can objective tests also play a part? r l fie case for introducing 
the purely objective test rests on the assumption that writing 
involves a complex combination of skills which may to a certain 
extent lx? isolated and tested separately. Marking schemes for c*says 
frequently acknowledge the many aspects of writing when the)' 
make provision for the separate as>e>nncnt of such aspects as 
accuracy in the use of structure, use of vocabulary, selling, etc. It 
can he argued that the essay is not the most efficient means of test- 
ing such features, since the examiner is dependent on what the 
pupil decides to produce in order to make his ;wses>mcnt. A more 
searching and planned assessment could almost certainly l>c made 
in a number of separate objective test*. I he reason that such test* 
do not play a part in our examinations is the fear of their elTcct 
on teaching. 1 lie use of objective techniques alongside die more 
conventional essay t s j >c of question, however; would seem to merit 
further research and experiment. If accuracy were 1 a«w«scd hv 
objective techniques only, arid not in the essay, it would be interest- 
ing to sec if this had any marked clfcct on tire quality of the c*sa\s 
pioduccd in examinations. 

Although completely objective techniques do not \ct appeal 
in GCK or CSK tests of writing, a number of technique* are now 
King included which arc capable of King marked with a high 
degree e*f inter-judge' reliability. An example' of one such technique 
is die me of a series of questions on pictures which is included in 
some CSK examinations, and will appear in some of the cxjxri- 
inenta) k O* level examinations for pupils testing the Nuffield 
Schools Council materials. Such a technique JpjxMi* to offer a 
immKr of advantage*: it may l* irliably scored, it may K '.wed 
to sample a fairly wide' area of the sxllabu*. and its elTcct on teach- 
ing is likely to l>c Kn« fie ial As the pupils' inanely of the language' 
increases so the need for such tests diminishes. I here i c certainly, 
however, a c.we for their u*c at the level of first rxaminatic'ns. 

>\\) Sf-faKing 

Testing the pupils' ability to q»cuk the language i* generally 
.lccepteel a* King the ino*t difficult <'f all the language toting ta ke. 
In addition to many prohhn;* faccel in testing writing there ate 
the problem* of jndi\ : dml l« at d the limited amount of lime 
available fen a^c^ing each pupil. 



Piin'lcur lms slated that the 'problem of objective testing of 
speaking ability has largely been overcome in recent years' 1 . By this 
be presumably means that it is |>ossible to construct a test of speak- 
ing which can be marked with a high degree of icliabihty. Unfor- 
tunately there must remain a suspicion that this reliability may 
well be achieved at the expense of validity. Most published tests of 
sj)cakiiig base their assessment on such a limited sample of the 
pupil's performance that it is hard to feel convinced that the tests 
arc valid. 

lhcrc docs, however, seem to be a need for semi-objective 
tests in addition to the ‘free conversation' which already exists in 
oral examinations. In the 'O' level examinations for Nuffield /Schools 
Council pupils question* on pictures, narration of events presented 
in a sequence of picture*, and role-playing have l>ecn introduced. 
All thc^c techniques may lx* marked very reliably if a clear ly 
defined rating scale is devised. 

Another technique which some boards arc currently employing 
is the ‘hundred prepared questions'. An interesting aqxct of the 
'hundred picparcd questions' is that it appears to mark a radical 
departure from the CiC-K loardd normal procedure of avoiding 
any indication as to the linguistic syllabus on which the examination 
is baud. Hut it is unfortunate that in this cau a decision alxnrt 
syllabus content apjx\m to have been taken without f«r>t consider- 
ing a "tailed definition of the objectives for oral work. Had these 
objectives Ixen clearly defined there would study have been no 
need to list the actual hundred questions to Ik* answered. It seems 
doubtful whether diis will ultimately lie in lire IxM interests of 
oral teaching. 

Postscript 

One inifKutant effect of the introduction of objective testing his not set 
l*ecn mentioned, lire production of such tests is a lengthy and expensive busi- 
ness and a mnnlxr of boards have recognised the desirability of co-ojk rating 
in establishing an item bank. A recent report by the NTER* has shown the 
feasibility of producing an item bank for the uw of USE l*c\mh and teachers 
consti uc ting their own Mode 111 examinations. It i* reasonable to suppose 
that the degree of co-opcTation between examining (kmicIs will increase as the 
Treed is felt for more sophisticated and reliable testing procedures. 

1 PimsTcur. P . 'Ttstire *ortirn Tarcuacr Inn =ir*\ ValtJman. A . r<b?« r. Turds n 
fecrkoir. McGnw-lIill. New York. 1°*^. p 
: SkutruV. t . S . /fen franb’rf. Na’Jot at foundation ft (duca'i't a.1 Rf't.uch. |tyV* 
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The use and efficacy of 
continuous assessment 



B. G. PAKMBR 

In proportion as dissatisfaction with traditional techniques of examining 
has grown, interest has increased in the contrasting technique of continuous 
assessment. Some six years ago, CSK became the first public examination in 
this country serious!)- to include teachers* assessments, although the college* 
of education probably have the longest cxj>crience in the field. I lie latest mani- 
festation of interest is contained in the PropOsafs for the curticuium and 
examinations in the sixth form' where it is stated that the new examination 
‘should allow appropriate combination of external and internal assessment, 
including the assessment of course work*. 

Efficacy of continuous assessment 

The efficacy of a form of assessment (efficacy is more logically discussed 
before use) may be examined vindcr two main headings : 

(1) Reliability 

Continuous assessment is necessarily subjective; examinations 
arc jx>tcntially more reliable csj>eeially with 'objective* method*. 
I he reliability of CA can be greatly increased by introducing other 
assessors to counter individual fluctuation and jxrrsonality factors 
— by internal moderation. In public examinations, to ensure com- 
parability of standards, external moderation is also necessary. 
Reliability i* likely also to increase with the cxjvencnce of the 
teacher in this form of assessment and with the length of juried 
over which it takes place. 

(2) I c tidily 

In the most complete form of continuous assessment all 
clement* of the course may l>c represented in the final result. This 
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docs not necessarily ensure a valid assessment. The teacher needs 
to define his objectives with some exactness and then to gauge what 
is being measured and in what proportions. Examinations set by 
rcsjKmsiblc bodies are likely to lx- reasonably valid for their own 
examination syllabus, but may be much less so for individual teach- 
ing syllabuses. 

Adi outages 

The major advantages of CA as opposed to examinations seem to be 
'.hat it is flexible and can eater for individual dilTcicnecs in schools, teachers, 
children and syllabuses; that the pressure of work or ‘assessment motivation’ 
is spread evenly throughout the course and not related to the last few months 
or weeks. CA also avoids three inevitable disadvantages of syllabus-content 
examinations, and at least one other major potential disadvantage. It avoids 
arbitrary sampling since the whole syllabus can never be examined in the 
limited time of the examination; it avoids the intense psychological pressure 
created by examinations which is felt by a numticr of candidates (and which 
produces unrcliaM? results, and can also inhibit learning over a longer period 
of time). It also reduces the jjossibility of no result at all through illness, physical 
accident, etc., and may avoid the backwash of examinations which in practice 
lias uMially proved restrictive, and harmful IxmIi tot content and technique. 

(Examinations can, however, also be designed to have a l>cneficial back- 
wash, and in that sense may lead to curricular innovation, c.g. stress on oral 
skills in language examinations.) 

Dnad: ti vt ages 

Continuous assessment has certain disadvantages. The demands on the 
teacher arc much greater, lie must have more initiative in clarifying his objec- 
tives, greater objectivity in his assessment, greater diligence and care in bis 
record keeping. He will also sjxnd more time moderating the work of col- 
leagues. Malpractice (copsing and plagiarism) is easier to gel away with. 
Against this some form of sporadic testing, or in the ease of a long essay or 
project, a liia, may be the only defence. Improvement /progress by students 
is not always adequately nwaided. Differential assessment can mitigate this 
(i.c. work early in the course could count for a lower proportion of the tola! 
assessment). 

lherc arc othrr jotrnti.il disadvantages which t have not experienced 
in practice. Ibe teacher student relationship niav lx* adversely affected by the 
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former's perpetual role as assessor being emphasised. Partly because of this the 
student may be under psychological strain throughout the course. Because the 
student dare not risk a low mark and thereby ‘lower his average*, learning may 
become less ambitious and more convergent. 

Such ail analysis of continuous assessment suggests that we can 
reasonably think of assessment as falling within the competence of the teacher, 
but supplemented or complemented by external niodeiation and examinations 
wl crc necessary, although often we start from the oppo>itc end. It is certainly 
clear that most of the deficiencies of IjoiIi methods of assessment can be over- 
come by appropriate combination- -the prO|>oi lions must clearly vary at least 
according to the level, the subject being learned and the objective* of the 
course. 

In assessing languages, it is an interesting exercise to start from the 
assumption of no examinations, and then to make a ease for each one piojxwcd 
~ e.g. if it is an objective of the course that a child should lx 4 able by the end 
to ask his way in Trench in Paris and understand the reply, then a certain type 
of final oral test is justified, and so on. 



Prrsrtit us r of continuant a*sc**meut 

So far in this country there U relatively little general <\p<rirucc of con- 
tinuous assessment, and even less of its application to langur gev 



(lj liy CSK board* 

I roni the outlet the ( »(IK t\po examination was to be ci ideally 
re-appraised and other |<o*dbililic < encouraged : 

■ihr ui.iin puijKwr of *:i\ «\»Um of f \air.i:i*'i< t«« i« lo «rivc a* in ?Mtr«i- 
fur nl i o riir»»uif ability ant! ac l ie \crnrM. If. however, lie f XMidraiiom <o;*io 
st>V)y of unsren pap<M and pm lira] u u« Kl al a *p<cific h.otm r ,t of tine. thr\ 
measure ore a<prci only of total pM form an<c and iht y do so to ihr di«ad\ aruacc 
of ondidifts who. thou cli able. a T c had examiner* ot who wrrr. by reason of 
sicVrrss, rrcrrt <ruiy to the di-trirt or some i«1 fir r <au*r t*\or.d ilait < or, it oh at 
a disadvantage at the timv thry *at for the i aartiinanoTi 

At, of ahilin. aid arlncu ir.r r.t cam ihr whole duiA’iori of I 1 r 

cO’jho hadin'? up to the examination wou’d. if taVm in conjuration with 1 1 . c 
r ( * j f ! « of tlr < -a ami nation rot erh do cr.-atrr ju«ticc to r a?> cardidaliv 

f lit would fnaf'lr otl,M qua’irir; to If ir,fA«ur<.d such a* rl c rracir.i’ir-n. 

in:t : afivf ard p f r«r\r *r«r io<mi\ for |p p’a'ivr c and tor: pVtkn rf a p:oj<<l 
requirirS yorre irv^Miranon or ir-V-M ton'ili, 1i wa« »hrr* dfodul that a.i 
*« r«*rrf rt of 107 r ? a<prr t r f < our «r wor l would I p< »miti« d an 1 ink < d r r < r»i ’ .« cr d 
in i hr ( Sf. L'ft tlVi'.ia f leniMr u /> rri f{> fvlcf; -rr / r '5°. p? 



Under Mode I (as well as under Mode III), there is wide use of various types 
and degrees of GA (with variation between board and board, subject and sub- 
ject). With two boards the teacher’s assessment is taken into account only for 
upgrading a candidate who has done unexpectedly badly in the examination. 
In languages CA is much less common. Information from twelve of the fourteen 
board* reveals that in only two and a half (half being the North section of the 
East Anglian board) is a definite proportion of the marks eiven for course work. 
In two others project work may be used for upgrading. 

(2) In colleges of education 

The longest experience of CA is probably in the colleges of the Bristol 
School of Education, but many other colleges now use course work in making 
the final assessment. There is a wide variety of practice, Iti Berkshire College 
of Education, out of sixteen subject areas five have no examinations at all, one 
lias no written examination; the majority have a combination of course work 
and examinations, but not all in the same proportions. The languages (French 
and German) have GO per cent course work and dO per cent examination, but 
fourth year B.Ed. studies arc assessed only by examination. 



Future use 

'Flic use of CA seems likely to go on increasing csjK'cially in the assess- 
ment of languages. If examinations can be accepted as complementary to CA. 
debate might be directed towards, for example, an identification of the specific 
objectives of courses at various levels and the isolation of those objectives which 
can be assessed cmy by end of course examination. In discussion of particular 
problems such as the assessment of oral contribution and skills, the place of 
diagnostic testing will no doubt emerge. 
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CHAPTER 8 



Examinations in English as 
a foreign language 



G, E. PER REX 

Clearly, examining English as a foreign language is big business, and one 
which has been going on for a long time*. Since in the world at large English 
is the most widely taught foreign language, it is not surprising that the range 
of examining techniques employed is very large indeed — from the highly 
sophisticated to the very naive. In this particular context, the examining of 
English is fundamentally diff erent from the examining of other languages, for 
we are considering our own language as learned by foreigners and not foreign 
languages as learned by English speakers. Moreover, in this ease the examiners 
themselves arc native speakers, operating as it were from a home base, and not 
as in the case of most foreign language examiners in Britain, foreign speakers 
of the language concerned. 

Home-based examinations in English as a foreign language, then, are 
not necessarily linked to a particular educational system and its associated 
values, are applicable primarily to those who want to take them as evidence of 
their ability to use English, and to some extent arc competitive with each other. 
It follows that they arc likely to be far more linguistically based than educa- 
tionally based; the examiners are certainly not cxjjerts in the educational back- 
ground of all those foreigners who sit for their examinations, but they do claim 
to be experts in the language— if only because it is their own mother tongue. 

We all think we know a lot about our own language, and indeed we 
do — .although not always explicitly in those terms in which it needs to be 
presented to foreigners, Engl id i too has been the subject of a great deal of 
advanced linguistic research*. As the linguists will tell us, it is no easy matter 

1 Ste Note, p. 64 

* It should be noted that a number of British universities offer advanced postgraduate 
courses for teachers of English as a foreign language, often with a high content ol 
theoretical and applied linguistics. There is almost nothing comparable for the teacher 
of other foreign languages 
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to test English as a foreign language, simply because \vc now know so much 
about English. The matter is made more complex because English Is mostly 
taught abroad by foreigners who, if often deplorably ignorant about the lan- 
guage, nevertheless seem to be quite clear about what they want to teach, which 
is not always the same as we would like to test. We can imagine a similar situa- 
tion if the French taught in Britain by British teachers were examined by 
Frenchmen living in France. Jf so, no doubt we should be critical, but \vc 
should also be at a linguistic disadvantage in making some of our criticisnj. 

In this situation there arise overtones of national responsibility not 
associated with examining foreign languages in Britain. The English language 
abroad is often known (and probably judged) by examinations which are set in 
Britain, certainly as far as adult learners arc concerned. Thus Brilisli examining 
authoiitics which set papers in English feel that they arc, as it were, authenticat- 
ing the language, and perhaps the culture, of their own country to foreigners. 
British-based examinations of English as a foreign language are inevitably seen 
as a projection of Britain abroad. The foreign candidate expects his meed of 
British ‘civilisation* in such examinations — exported direct from Britain — and 
the British arc often very ambivalent al>out providing it. At any rate the British 
argue a great deal among themselves about the nature, consistency and appro- 
priate evidences of British civilisation which should be exhibited abroad in their 
examinations. 

1hu5 the choosers of set books for literature papers arc often in a fix. 
Quick to recognise (as liberal and progressive academics should) that Britain 
has developed a new relationship with the rest of the world since 1945, noting 
her less dominant and rather more' convivial role, they guiltily realise that much 
of that very English literature which foreigners appear to like reading most is 
riddled with imperialism, xenophobia, racialism or culpable self-confidence in 
the superiority of Britain. (When selecting French, German or Spanish texts, 
the British examiner has no such moral problems.) Jbe conflict between the 
claims of the classics and of a modest presentation of the contemporary scene 
can be quite agonising. 

Examinations in English as a foreign language have a long history. Ibc 
Cambridge Proficiency Examination was first established in 1913, and the 
Cambridge Ixnvcr Certificate in 1939. both examinations being designed 
primarily for non-Englisli shakers in Europe, T he examinations set by the 
RSA in English as a foreign language were originally deveIo[>cd to meet the 
needs of foreign students living in Britain, although now they arc extensively 
taken overseas, ?.nd are set at three levels. Examinations in English as a foreign 
language arc also administered by the Pitman Examinations Institute and the 



LCC. The latter body attempts to prov ide papers which are concerned especially 
with English used for commerce. All these examinations include tests of spoken 
English as well as tests of reading and writing. The oral tests vary in their 
sophistication and power. 

There is, however, an even longer history of examinations in English 
which have been taken by non-English speakers in colonial and ex-colonial 
territories outside Europe. Most of these examinations, as far as their English 
language papers are concerned, have been almost unmodified versions of 
examinations originally set for pupils in British schools. Thus they were designed 
less to test English as a foreign language, than to assess those skills considered 
appropriate to native speakers of English, They have therefore had a strong 
literary bias and a language content which for a foreign learner can only be 
regarded as curious. Until quite recently they included no oral test. The first 
overseas candidates for the London University Matriculation English papers 
sat (in Mauritius) in UU>5, while the Cambridge Local Examinations were first 
taken in the Straits Settlements in 1891 and in Lagos in 1910. Since then the 
Cambridge l>oard’s Overseas School Certificate, ; ts GCE and its many local 
derivatives have largely moulded the style and manner of English teaching in 
schools in great arras of Africa and Asia, with the general agreement and 
co-o)>oratjon of the local teachers. It is sometimes claimed that in countries 
where English is u^d as a medium of school instruction (although it must first 
be learned as a for n language), it thereby changes its status from a foreign 
to a second language «,nd should be examined by rather different methods than 
those appropriate in, say, France or Germany. Ibis may be true, but tire dif- 
ference should pci haps lie rather in varying the cultural content or context 
than in modifying tiro linguistic criteria or techniques of examining employed. 
Quite apart from the influence that these examinations have had on the teach- 
ing of English in schools, more recently certain GCE ‘0‘ level English papers 
have been taken by very large mmilxrrs of private candidates overseas, (in 
January 1989 for example, about 35,000 candidates sat for the London Univer- 
sity GCE ‘O’ level English paper.) Examinations made in England have an 
obviously high status overseas, supjxutcd to some extent by their value as a 
qualification for overseas students who wish to enter British universities or 
technical colleges for further training. 

Two recent developments deserve s|*cial notice. Three Nears ago the 
RSA established its certificate in the teaching of English as a foreign language, 
primarily designed for foreign teachers of English. In this examination, papers 
are set on the methodology and content of English teaching, but candidates who 
cannot produce external evidence of a previous good command of English may 



have to take a preliminary paper in the language. More recently Trinity College 
of Music has also established a Teacher's Licentiate Diploma in English as a 
second language which includes a test of oral English as well as one of teaching 
methodology'. In view of the importance of oral tests, those recently introduced 
by TCM arc of special interest. These new examinations in spoken English a - 11 
a second language can be taken in twelve grades, ranging front a level appro- 
priate to very young children to one suitable for adults. It is intended that the 
different grades can be taken at yearly intervals, thus covering up to twelve 
years of learning English. The examinations arc supplied by detailed syllabuses 
for each grade which provide far more gui dance than is normall) the ease in 
any language examination. There is nothing parallel to these Trinity College 
examination* generally available to student; of other foreign languages in 
Britain. 

A common criticL f most language examinations is that their 
syllabuses are inexplicit, un i\*h v ‘ or non-existent. While this applies to many 
examinations of Kngwh s;. ? S.i igu language, it certainly does not apply to 
all, Partly no doubt because o.my of these* examinations arc set in Britain, to 
be taken in many cJilfc rent countries, and because candidates are prepared by 
many different kinds of teacher, many of whom are themselves foreigners, 
English syllabuses lend to l>c more explicit than those provided in oilier foreign 
languages. But because the techniques of testing used have to be acceptable 
to the same wide variety of teachers and candidates, they tend to l>e conserva- 
tive. Taco’ validity of question and content rates high in such circui iMances, 
but face validity is not always a good guide to the construction of good examina- 
tions or good tests in languages. 

A number of examinations in English have developed and now employ 
objective and semi-objective techniques. These have been used quite extensively 
in the Cambridge Proficiency Examination (in the Use of English paj>cr), by 
the I .CXI and by others. 

One social as|>cct of English testing (as distinct from public ‘examining), 
deserves a brief note. In both Britain and the United States vety large mnnbcis 
of overseas students seek entry to institutions of higher education, and special 
English tests of proficiency are needed, which can l>c taken in overseas countries. 
This has given rise to valuable research on Iroth sides of the Atlantic, particu- 
larly into the problems of testing advanced proficiency in the use of a language. 
Some of the techniques evolved to test students’' practical ability to use English 
for purposes of learning vocational subjects might well be of interest to examines 
of other foreign languages at other levels. 



Nothing has been said about the many English examinations which are 
set and administered in countries throughout the world by foreign authorities. 
There are of course a great number of these simply because English is the most 
widely taught foreign language in school and out. Advanced techniques of 
examining are often used and some deserve careful study; more international 
interchange in this field would be beneficial, not least to British examining 
bodies. 



NOTE 



Entries for some examinations in English ns a foreign language 



Totals 



1968 



Cambridge Examinations in English 
Certificate of Proficiency Loiter Certificate 



14.371 



37,481 



51,852 





Royal Society of Arts , English for foreigners 




1968/9 


Stage l Stage 2 Stage 3 

4,000 3,804 1,030 


8,834 




Trinity College of Music, Spoken English 




1969 


Grades 1-12 


1,8-16 
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CHAPTER 9 



Summary of discussions 



The papers presented obviously introduced issues beyond the problems 
of setting, marking or administering examinations, more particularly that of 
die effect of examinations on teaching aims and techniques. One of the tradi- 
tionally accepted roles of school examinations in Britain has been to provide 
general control over the curriculum, and indeed over the content of the courses 
taught in secondary schools. In some subjects the examination syllabus is the 
school scheme of work for the one or two years preceding the examination. In 
others an interpretation of the expressed examination requirements is worked 
out in detail by the teachers and the schools. There arc established objections 
to both approaches. A rigidly controlled and detailed examination syllabus, 
designed ar . imjKiscd as it were by a central body, is regarded as a straight- 
jacket by sonic; while a loose description of general aims is regarded as too 
vague to secure efficient teaching by others. To whichever side we incline, it 
is quite clear that since examinations arc measuring instrument', they cannot 
be efficient (or efficiently prepared for) unless we have a clear notion of what 
they a f c measuring. Whether provided by examining boards or by schools, 
detailed teaching aims and objectives arc essential. 

In recent years the materials and techniques used for foreign language 
teaching have radically changed. This has not resulted from changes in examina- 
tion requirements but from new fashions, attitudes and insights into language 
learning. The shift of emphasis in classrooms from learning about languages and 
their associated literature, to learning to use languages lias led to a general 
desire for greater precision in describing' exactly what pupils arc expected to 
be able to do at the end of courses, but so far GCK f>oards have been cautious 
in committing themselves to such descriptions. 

GCE ' O' level 

The traditional imprecision of examination syllabuses probably exerts a 
conservative influence not only on the techniques of examining but on die 
methods of teaching employed in schools. Because no detailed statement is pro- 
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vidcd of exactly what candidates arc expected to have learned, teachers dili- 
gently search past papers not only for indications of die standard of performance 
required hut for likely questions which can be anticipated in their teaching. 
Departures from traditional methods of teaching or from established course- 
books may indeed app nr prejudicial to good examination icsults. A closer 
definition both of general aims and of teaching objectives to be realised hi the 
classroom might well en;ourage greater freedom of teaching techniques, albeit 
within defined limits. Specifications of what should be expected of pupils at 
different ages and stages would also be helpful. 

Vocabulary as a measure of what should be known by candidates lias, 
when used by itself, severe limitations. ‘Knowing words’ does not equate with 
knowing a language. Hut iti view of current uncertainty about the range of 
vocabulary which may be expected in examination papers, some guidance by 
examining Ijoards would be valuable. (In French, Ir fraufai: fondnmcnlal- 
suppoi ted by additional lists— might l>c a useful guide.) Apart from soim 
description of the linguistic content in the syllabus, a statement of the level of 
skill required would be helpful, particularly if comprehension of speech is to 
be tested. Teachers would find it useful too. to know the importance of accuracy 
in pronunciation (what tolerances arc permissible?), in salting and in gram- 
matical correctness in relation to the over-all proficiency required. Some skills 
of clear value* for example that of rapid silent reading for information, might 
be encouraged by providing appropriate tests. At present some techniques med 
for testing coin prehension of written language actively discourage the develop- 
ment of high reading speed by emphasising the close study of individual words 
at the expense of the co-ordination of contextual information. 

Greater flexibility of teaching* permitting a choice of materials to suit 
the varied interests of teachers and pupils, might be encouraged by requiring 
only limited coinpulsoty studies, and ofTciing a wide choice from a nninWr of 
optional units covering different ranges of language or centres of interest. 71iis 
would require a greater variety of published teaching texts, and no doubt pub- 
lishers would require, l>cforc marketing them, some assurance of quick and 
numerous sales. 



GCF VI’ far! 

It seems clear that present *A’ level syllabuses (in some respects much 
more closely defined than thoc at ‘O* level) are too narrow' in their scojie to 
ensure work of equal interest tn all those who may take sixth form coutscs. 
(Some sixih forms require only one year courses licyond 'O’ level ) Again it 
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might be valuable to limit the compulsory elements, and to provide a variety 
of alternatives to the conventional literature requirements. Papers in ‘civilisa- 
tion 1 at present tend to be unattractive, but papers winch emphasise contcm* 
porary intciests. some technical, might well be popular. They would need 
syllabuses which define their linguistic range as well as their content. 

Schools undoubtedly tend to slick to the known — the literature course 
— because of the widely assumed value of classical texts and their established 
teachability. Hut to guarantee safety in passing examinations, crude simplifica- 
tions of sophisticated literary values may have to be presented in classrooms. 
Literature should certainly remain as one of the options, and the prose- - so 
often attacked — also lias considerable value as an option. It is, however, doubtful 
whether formal translation into English should remain at all; an English sum- 
mary of a text in a foreign language would be more widely acceptable and could 
well encourage wider, faste*' and even more intelligent reading. If a literature 
paper i/ taken as an option then to require at least one essay in the foreign 
language on a text which has been studied in detail seems appropriate. Wide 
and rapid reading should he encouraged, although literary classics arc not 
necessarily the best material on which to develop this skill. 



CSK Mode lit 

There arc obvious difficulties in ensuring that Mode HI papers are 
comparable in standard to those set in Modes 1 and II. DilTcrent Ixxuds have 
devised different methods to keep them in line. In this field the experience of 
the Schools Council Modern Languages Project in establishing a common paper 
acceptable to several lx>ards, to be scored according to agreed principles, seems 
very relevant, especially in connection with techniques of testing listening and 
reading comprehension. 

Mode III permits teachers to decide their own teaching objectives and 
the relative weighting to Ire given to various skills; an examination syllabus as 
distinct from a school scheme of work is, in these circumstances, inappropriate. 
Hut if the schemes of different schools appear to require different levels of 
achievement, moderation may l>c needed; if the scheme of an individual school 
implies a lower level of achievement than is required by Modes 1 and IL a 
board may indicate that candidates taking this Mode 111 arc unlikely to bo 
able to achieve a grade I pass. 

If equality of standard with (»CK ‘O' level or with Modes I and 11 i* 
sought, srhools offering Mode HI may be tempted to concentrate on c camming 
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written skills, which would destroy sonic of the advantages of using Mode HI, 
The comparatively small number of candidates taking Mode HI (about 10 
per cent of the total) indicates that its opportunities arc not yet being fully 
exploited* in some areas it is not offered at all. Rome teachers are sull reluctant 
to employ newer techniques for testing, e.g, a tape recorder for assessing oral/ 
auditory skills. 



Non-school examinations 

hike those of CSE Mode III, the language examinations of the Ordinary 
and Higher National Diplomas (Business Studies) arc internally set and sub- 
ject to external approval and scoring, but there are the following differences : 

(i) The joint committee provides a model syllabus, which is accepted 
by most colleges. 

{»} A pass in the oral test is obligatory. 

(iii) Previous course work is assessed. 

Equivalent levels of achievement in different colleges could be encouraged 
through more contact between colleges and by circulating copies of papers set 
or of recorded versions of oral tests. Comparative statistics of results could also 
be published, 

In non-school examinations oral tests arc emphasised and these often 
require a high level of fluency and expression. Because of the varied vocational 
interests involved, however, it is not always easy to find oral examiners who 
frosscss the necessary* combination of high fluency and a knowledge of the 
technical terms likely to be required to discuss the particular interests of can 
didates. Although using a language laboratory to present questions and record 
candidates’ answers may appear to help, it must be remembered that scoring 
a taped oral tost is slow* and expensive. If live examiners are med, recording 
facilitates external moderation by permitting sample re-scoring. 

Oral examining 

Examinations of oral ability require a syllabus no less than examinations 
of reading or writing si.'lls— it is quite insufficient to provide a vague statement 
of aspirations or merely iO indicate certain social situations within which can- 
didates should be prepared to talk. Standards of perfe nuance icqmrcd could 
be cla ified by making and circulating recordings of model or.il tests together 
with an analysis of how such examples arc scored. It is felt that the one hundred 
prepared questions, the listing of twenty to thirty topics for preparation and 
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study, or the use of project work as a basis for oral testing could all be helpful 
in defining in practical terms not only objectives but the standards of oral skills 
required. 

In terms of the time, travelling and subsistence expenses of external 
examiners (not to mention the loss of teaching time involved in some institu- 
tions) traditional methods of conducting oral tests are expensive. The return 
for this expenditure is at present very limited. When the examiner himself par- 
ticipates in exchanges of speech at the same time as he is attempting to score 
the responses, tests can be particularly unreliable or become virtually non- 
dhcrirmnatory. T he reliability of scoring v:>r s greatly according to the nature 
of die test materials, the qualities which arc selected for scoring, the extent to 
which over-all performance is notionally sub-divided into sub-skills and the 
weighting given to them in scoring. Such weighting should not necessarily be 
the same at all levels of proficiency, or in all languages. (Tolerances of pro- 
nunciation may be wider in, say, German than in French, and in English 
perhaps widest of all.) Unstructured tests of ‘conversation’, requiring a series 
of marks based on the assessment by one examiner of several supposedly dis- 
crete skills, simultaneously employed, can only become more unreliable as 
greater discrimination is attempted. An 'impression' mark on a simple pass-fail 
scale may be much more reliable, but this only raises new problems of validity. 
When results have to be graded, as in C$E, it is difficult to ensure both good 
reliability and adequate discrimination. 

Proposals to increase the over-all weight given to oral performance in 
GCE examinations could, unless the tests used were much more effective than 
at present, lead to those pupils seeking university entrance being placed at 
greater risk. T he present low correlation between results in oral and written 
tests suggests that it is wiser to submit scores for them independently rather than 
to combine them. 



Research by examining boards 

Hie care taken by examining boards to ensure comparability of stan- 
dards in GCE papers, and similarity in standards between Modes \ t 11 and III 
in CSE papers, is welcomed. Hut the details of such work arc not widely known 
ouUide those immediately professionally involved. An air of Olympian mystery 
is attributed to the delibera'ions of boards by many teachers, parents, and can- 
didates, which could perhaps be disjxrscd by freer information. It is realised 
that reports of all their detailed work might not be suitable for wide publica- 
tion; but even if marks are confidential, the processes cf arriving at them need 



not be. It is believed that teachers would benefit from more information and 
that teachers in training should be taught not only the principles of language 
testing but more about the administration and control of public examinations. 
This would improve confidence in results and lead to better participation by 
teachers in developing new techniques. 



Objective tests 

The use of objective tests of achievement requires the exact definition 
of aims and a precise analysis of the content of the course of instruction to be 
tested. It is clear that objective tests can offer great economy in administration 
and scoring costs. But their use also helps considerably in the task of clarifying 
objectives and establishing the relative value of the multiple skiils which con- 
tribute to language proficiency. Apart from their use in making comparative 
assessments of performance, they can be of great value as a research tool to 
tell us more about language and language teaching. At present objective tech- 
niques arc ostensibly most suitable for assessing the receptive skills of compre- 
hension, but arc less manageable in measuring the piaduction of speech and 
writing. But just as an unstructured conversational test of speech can be wildly 
unreliable, so the essay-type test of writing skills can be unreliable, uneconomical 
and inefficient. The backwash effect or non-crcativc quality of objective tech- 
niques is often criticised, but controlled methods of eliciting written responses 
can sometimes be scored scim-objcctivcly, c.g. by requiring written responses 
to pictures us:d as stimuli, the completion of a narrative sequence or the com- 
position of structured letters. It is also jK-rfectly reasonable to provide a detailed 
examination syllabus and to employ objective tests to assess whether it has been 
properly covered. 

The pro]>osal for an item bank of pre-tested materials, available to 
examining boaids, is not new and remains attractive; although in our present 
state of imprecision about syllabuses, it might have to be associated with a bank 
of aims and objectives. 



Continuous assessment 

If misunderstood, the notion of continuous assessment can easily 
encourage continual testing, which has obvious disadvantages, lxmg term 
studies of the rcliabil* y of continuous assessment as compared with that of 
conventional jxriodic «ing or examinations are required, as well as clarifica- 
tion of its effect on st dent motivation and on studcni-tcachcr relationships. In 
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schools, the status of the public examination v, such that it could not suddenly 
be replaced, bearing in mind the attitudes of parents and employers. If con* 
tinuous assessment is used in schools, problems arising from changes of school 
or changes of teacher have to be adequately dealt with. 

In universities, where continuous assessment has in the past been used 
'ittlc in undergraduate work but much more in graduate work, there arc signs 
of a change round The newer universities rely more on assessments of under* 
graduate course work, while at the second degree level examinations are 
becoming more common. 

ResfAjnsibility for assessment should as far as possible be spread between 
teachers, not only to increase reliability but to increase student confidence in 
the ratings given. Clearly, too, students must know these ratings, if not con- 
tinuously then at periodic intervals. 

The ranking or assessment of candidates given to examining boards by 
teachers is a form of continuous assessment, which appears to favour pupils 
in comparison with their later examination results. While the role of continuous 
assessment in the obviously practical subjects (such as arts and crafts) is clear 
because of the nature of the tasks learned, its true value in relation to foreign 
language courses awaits the careful listing of objectives arid tbc ordering and 
weighting of the skiils involved. The more foreign language achievement is 
understood iti terms of skills rather than knowledge, the more appropriate 
continuous assessment becomes. 

English gj a foreign language 

Examinations set and administered from Hr i tain and taken by candi- 
dates from many countries both at home and abroad arc subject to multiple 
feed-back. A very large ]>roj>ortion of candidates for the Cambridge and RSA 
examinations in English take them in Britain. ’! lie paj>crs set must therefore 
satisfy both foreign and British teachers of English, of widely dilTciing stan- 
dards of eonij>ctcncc both in English and in leaching skill. The variety of 
demand for a J provision of English has led to interesting and practical develop- 
ments in the techniques of language examining. These arc not always wcli 
enough known to those concerned with other languages. It is felt that from 
the [joints of view both of examining and of teaching techniques theic could 
well be a greater interchange of information between teachers of English as a 
second language and teachers of other foreign languages. 
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Current research 



The following extracts from GILT Research Rcgistci describe 
research in progress in Great Britain. 

727 Mrs C. Burstail, National Foundation for Educational 
Research, 79 VVimpolc Street, London \V1. Associates: 2 
research assistants, 2 technical assistants. Sponsor: Depart- 
ment of Education and Science. 

The French project: an investigation into the teaching of French in primary 
schools. The project will present an independent evaluation of the pilot scheme 
for the teaching of French in selected primary schools. A longitudinal study is 
being carried out of 2 consecutive year-groups of pupils (about 12,000 children). 
Suitable control groups have been set up. Pests of proficiency in French (listen- 
ing, speaking, reading and writing) have been constructed by the project staff. 
IPic experimental sample will l>c followed through until the end of their second 
year m the secondary school. Attitude scales, proficiency tests, questionnaires 
etc. will be constructed as required. The study is being continued with a third 
year-group (see no. 902). Date begun: May 1964. Progress reported: in French 
from eight: a national experiment (first report) by Clare Burstail (see Appen- 
dix 2); next report to be published about September 1970. 

764 C. V. Russell, University of Ix>ndon Institute of Education, 

Malct Street, 1 ondon \ VC- 1 . 

A comparative study of examination techniques in modern languages in the 
Federal Republic of Germany and the United Kingdom. (The rrscarchcr 
attended as an observer the conduct of the Abitur examination (oral section) 
in eight schools in Rheinland -Westfalen during May J969.) For degree of MA. 
Date begun: July 1967. 

835 E. F. Chapicn, 27 Birch Hall Lane, Manchester MI3 0XJ. 

(Research at : Faculty of Edi ration, University of Man- 
chester, Manchester Nil 3 9PL.) Sponsor: Social Science Re- 
search Council. 

Construction, validation and use of an F.nglish proficiency test battery. 

(I) Construction and validation of the battery, for non-native shakers of 
English who intend to pursue diploma or degree studies in British universities. 
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(2) Use of the battery to determine what proportion of undergraduates and of 
postgraduates from overseas are probably under-achieving as a result of lack 
of proficiency in English, lest battery has been administered to overseas students 
at universities and assessments of Knglis.i proficiency have been obtained from 
tutors, In order to estimate the proj>ortion of students who would benefit from 
remedial English courses. For degree of PhD. Date iegun: October 1967. 

902 Mrs C. Burst all, National Foundation for Educational 
Research, 79 Wimpole Street, London \VJ . Associates: 

1 research assistant, 1 technical assistant. Sponsor: Schools 
Council. 

The third cohort study. An extension of the evaluation of the teaching of French 
in primary schools (rce no. 727). The main aims are : to carryout a longitudinal 
study of a third year-group of 8-year-old beginners in French; to investigate 
further the teaching of French in small rural schools; to study in detail the 
factors determining less able pupils’ success or failure in learning French; to 
investigate at the primary level promising lines of enquiry developed during 
the evaluation of the secondary stage of the pilot scheme. Date bequn : April 
1968. 

903 P, Dyson, De La Salle College of Education, Middleton, 
Manchester. Sponsor: Modem Language Association. 

Oral testing in French (Modern Language Association research project). (1) A 
comparison of analytical and impression marking systems for oral assessment. 
(2) A comparison of different testing techniques for conversation (‘free’ con- 
versation as opposed to ‘guided’ conversation). (3) Can pronunciation and 
intonation be assessed along with reading? Date begun: October 1969. 

903 Dr \V. D. Halls, Department of Educational Studies, Univer- 
sity of Oxford, 15 Nor ham Gardens, Oxford. Sponsors: 

Council of Euroj>c, Gulbcnkian Foundation. 

Oxford/Council of Europe study for the ecafuation of the curriculum and 
examinations: sub-project : modern languages at the upj>cr academic secondary 
level : a comparison of the curriculum and examinations in the member states 
of the Council of Europe. This entails the study of official documents relating 
to aims, syllabus, curriculum and examinations; survey questionnaire; field 
work and personal interviews. Date beqi<n: November 1968. Progress reported: 
in report by \\\ D. Halls {see Ap|>endix 2). 

909 \V. Prescott, University of York, Mteklcgate House, Mieklc- 

gate, York S'Ol 1JZ. (Schools Council Modern languages 
Project : Evaluation Section.) Associates: A. Crispin, B. Park. 

U. Sankarayya, J. Simpson. Sponsor: Schools Council. 
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Research on tests and testing and examinations. (1) Production of achievement 
tests in French, German, Russian and Spanish for pupils aged 11-16 and (in 
French) 8-16 who are following the Nuffield /Schools Council materials in these 
languages. (2) Development of experimental ‘O* level examinations in modern 
languages. (3) On-going research and testing during production of materials. 
The tests produced will be published by the publishers of the Nuffield /Schools 
Council course materials. Details of the examinations arc already available. 
Date begun: 1968. 

910 Welsh Joint Education Committee, 30 Cathedral Road, 

Cardiff, Glamorgan. 

Research to establish an interim model of standard spoken Welsh to form the 
basis of all aspects of Welsh language teaching in schools, especially the teach- 
ing of Wehh as a second language. The work, is being carried out by a panel 
comprising language specialists and teachers. l)ate begun: February 1965. 
Progress reported: Cymraeg By tv: Rhifyn 1 1 published June 1967 by the WJEC. 

911 Welsh Joint Education Committee, 30 Cathedral R^ad, 

CardilF, Glamorgan. 

Compilation o( a fist of the basic sentence structure: and vocabulary of Welsh. 
Examples of the structures listed will be provided; the list is intended to cover 
the structures and vocabulary to be known by students of Welsh as a second 
language who are candidates for examinations in Welsh {GCK '()’ level and 
GSE). Dote begun: February 1965, Progress repotted: Cymraeg By :v: Rhifyn 11 
published Jv^e 1967 by the WJEO. 

914 T, A. Garter, Language Centre, University of Southampton, 
llighficld, Southampton SG9 5NH, Associates: Mrs J. Russ. 

Miss A. Lcyland, l)r K, Zobcl (University of Southampton): 

M. Taylor (Bell School of Languages, Cambridge). Sponsor: 

Office du Baccalaurcat International, Geneva. 

Research on multiplc-choice and other objective tests of language acquisition 
at university entrance fete!. Preparation of tests, item and battery analyses of 
validity, reliability and backwash effect on teaching. Correlations with national 
examinations in several Eurojxran countries. Date begun : January 1967. 

9/5 T. S, Percival, Department of Education, University of 
Manchester, Manchester M13 9PL. Sponsor: Schools 
Council. 

Comfrucfi’on of a standardised battery of French tests, ’bests of listening, read- 
ing, writing and speaking arc envisaged, for use below CSE/GCK level. A 
preliminary study was carried out, comprising development and standardisation 
of a listening test (details of this study are on file at CIL1'); the present work 
will be carried out on similar lines. Date begun: 1970. 
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Select bibliography 

This short list of books and articles relevant to the topic of 
the conference includes publications referred to in the text 
of the rcjjort, but not fully described there. 

Burstall, Clare : French from eight: a national experiment. Na'ional Foundation 
for Educational Research, Slough, 1968. (Occasional Publication 
Series 18.) 

Centre for Information on Language "l eaching : foreign language testing. 3rd 
edn, GILT, 1969. (Specialised Bibliography 1.) 

Christopher, R. : JMB/GCE: th e work of the Joint Matriculation Board. Joint 
Matriculation Board, Manchester, 1969. (OP 29.) 

Connaughton, 1. M, : 'The validity of examinations at 16-plus*. Educational 
Research, vol. 11 no. 3, 1969, pp. 163 -78. 

Davies, Alan, editor: language testing symposium: a psycholinguists approach . 
Oxford University Press, 1968. (Language and Language {.earning 21.) 

Department of Education and Science: The Certificate of Secondary Educa- 
tion. DES, 1968. (Reports on Education -17.) 

Department of Education and Science : Statistics of education 196 6 : vol. 2: 
GCE, CSH and school leavers. Annual, HMSO, 1968-. (l or 1962-65 
see Statistics of education 1962-65: part three. Annual. HMSO. 
1964-67.) 

Grey, D. : ‘A CSE examination in German*. Modern languages, vol. 47 no. 3. 
1966, pp. 104-7. 

Halls, \V. D. : Report on modern languages at the upper academic secondary 
level. Oxford University Department of Education, 1969. (Oxford/ 
Council of Eurojx v Study for the Evaluation of the Curriculum and 
Examinations.) (Mimeographed.) 

Hitchman, P. j.: Examining oral English in schools. Methuen, 1966. 

joint Matriculation Board: O-lcvel I.atin, French and biology: an enquiry 
conducted by the University of l.ondon School Examinations Council 
and (he Joint Matriculation Board into examination standards. The 
Board, Manchester, 1966. (OP 24.) 

Long, A. I. : 'An a Iter na tivc French ‘O' level examination’. Modern Languages, 
vol. 48 no. 3, 1967, pp. 105 8. 
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Lindley, D. V. : ‘An experiment in the marking of an examination. Journal 
of the Royal Statistical Society, scries A, vol. 124 pt. 3, 1961, pp. 285-313. 

Macintosh, H. G. : Objective testing. Associated Examining Board, Aldershot, 
1968. 

Mather, D. R., N. France and G. T. Sare : The Certificate of Secondary 
Education :a handbook for moderators. Collins, 1965. 

Modern Language Association: Modem language courses in the sixth form. 
MLA, 1968. 

Otter, II. S. : A junctional language examination: the Modern Language 
Association examinations project. Oxford University Tress, 1968. (Lan- 
guage and Language Learning 19.) 

Peterson, A. 13. C. : 'International exam/ Times Educational Supplement, no. 
2702, March 3 1967, p. 735. 

Proposals for the curriculum and examinations in the sixth form: a joint state- 
ment . . . Standing Conference on University Entrance and Schools 
Council Joint Working Party or\ Sixth Form Curriculum and Examina- 
tions . . . (December 1969). 

Schools Council : Examinations Bulletins : 

5. The Certificate of Secondary Education : school-based examina- 
tions: examining, assessing and moderating by teachers. HMSO, 
1965. 

11. The Certificate of Secondary Education: trial examinations— oral 
English . HMSO, 1966. 

12. Multiple marking of English compositions: an account of an experi- 
ment; by J. N. Britton, N. C. Martin, H. Rosen. HMSO, 1966. 

20. CSE: a group study approach to research and development ; edited 
by \V. G. A. Rudd. To be published by Evans/ Methuen Educa- 
tional for Schools Council, October 1970. 

Schools Council : Examining at 76 b: the report of the joint GCE/CSE Com- 
mittee of the Schools Council. HMSO, 1966. 

Schools Council : Working Papers : 

5. Sixth form curriculum and examinations. HMSO, 1966. 

6. The 1965 CSE monitoring experiment. Pts. 1-2. HMSO, 1966. 

19. Development of modern language teaching in secondary schools. 
HMSO, 1969. 

20. Sixth form examining methods , HMSO, 1968. 

21. The 1966 CSE monitoring experiment . HMSO, 1969. 

28. Kew patterns in six f h form modern language studies. To be pub- 
lished by Evans/Mcthucn Educational, October 1970. 

Scottish Education Department : Scottish educational statistics 1966 -. Annual. 
HMSO, 1967-, 
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West Yorkshire and Lindsey Regional Examining Board : CSE examinations 
1970: assessment in a nutshell. Rev. and reprinted. The Board, Sheffield, 
1969. 

West Yorkshire and Lindsey Regional Examining Board : Conducting an agree- 
ment trial: a working manual. Rev, and reprinted. The Board, Sheffield, 
1968. 

Wlialley, G. E. : The Certificate of Secondary Education: an account of the 
CSE examination , its organisation and methods. University of Leeds 
Institute of Education, I>ceds, 1969. 

Whitfield, Richard C. : ‘Improving examining at sixteen plus’. Educational 
Research, vol. 10 no. 2, 1968, pp. 109-13. 

Wilkinson, Andrew : Spoken English. 2nd impression, revised. University of 
Birmingham, Birmingham, 1966. (Educational Review Occasional Pub- 
lications 2.) 
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